Implementation Evaluation of HIV/AIDS Voluntary Counseling and Testing (VCT) Service at Public Health Facilities of Akaki Kality Sub-City, Addis Ababa, Ethiopia

Background

Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome (HIV/AIDS) is a severe and life-threatening illness that has had a major impact on public health worldwide. It damages and destroys the immune system, making the body more susceptible to other infections and cancers. HIV/AIDS has had a devastating impact on millions of people worldwide, particularly in developing countries.¹ Almost 33 million lives have been lost so far due to HIV infection, indicating that it is a major public health concern.²

In 2021, there were 38.4 million people living with HIV worldwide, 1.5 million new infections, and an estimated 650,000 AIDS-related deaths.¹ Among these deaths and new infections, 300,000 deaths and 730,000 new infections occurred in Eastern and Southern Africa, respectively. Among 4500 new infections (in adults and children) worldwide, 59% are in sub-Saharan Africa.^3,4 In 2019, there were 15,000 new cases of HIV infection and 12,000 deaths related to AIDS. Ethiopia is considered one of the nations most heavily impacted by the HIV/AIDS epidemic. Based on projections and estimates related to HIV in Ethiopia, the number of individuals living with HIV (PLHIV) is 609,349 by the end of 2022. Only 82% of PLHIV in Ethiopia have known their status.^3,5 It is currently estimated that one in every five people with HIV remains unaware of their status.⁶

Initiating the HIV counseling and testing (HCT) program is a significant milestone in achieving the Joint United Nations Programme on HIV and AIDS (UNAIDS) 2020 fast-tracking targets 95–95–95 testing for 2030. The first component of this framework aims to ensure that 95% of people living with HIV (PLHIV) will know their status.^1,7 HIV testing and counseling services are delivered through two main models: health facility-based and community-based. HIV testing and counseling services are offered through two main approaches in health facilities: Voluntary Counselling and Testing (VCT) and Provider Initiated Testing and Counselling (PITC).^8,9 VCT is considered a crucial component in prevention and care and has gained global recognition as a highly effective strategy for addressing both AIDS prevention and care. VCT differs from other HIV testing services in terms of pre-counseling, post-counseling, and risk reduction plans, which are not commonly applicable to other testing services.¹⁰ Although voluntary HIV counseling and testing (VCT) uptake has increased globally, there are still challenges in certain parts of the world, including a lack of resources, low national prioritization, low levels of awareness and access to VCT services, and overburdened health care providers.¹¹ Prior to the year 2000, only a small percentage of people affected by AIDS in Ethiopia had access to VCT (voluntary counseling and testing) services. Voluntary HIV counseling and testing continue to be crucial tactics for managing the transmission of HIV while also rendering assistance and care to those who test positive.^10,12 The VCT program’s components were explained through the use of a program-logic mode (Supplementary Figure 1).

To date there was no study conducted on the implementation status of VCT service in the study area specifically and in Ethiopia widely. Therefore, this study aimed to evaluate the implementation status of HIV/AIDS Voluntary Counseling and Testing (VCT) service in public health facilities of Akaki Kality sub-city, Addis Ababa, Ethiopia.

Evaluation Method and Materials

Study Site and Population

The evaluation of the voluntary HIV/AIDS counselling and testing program implementation in public health facilities of Akaki Kality sub-city in Addis Ababa, Ethiopia was conducted using a single case study evaluation design along with a mixed methods approach, from May 12 to June 12, 2021. Quantitative and qualitative data were collected concurrently.

As per the 2007 census, the population of Addis Ababa city was 3,627,934. At present, Addis Ababa is divided into 11 sub-cities and 119 woredas. The largest of these sub-cities is Akaki-Kality, which comprises 13 woredas located in the southeastern part of the city, about 20 kilometers from the city. The sub-city has an estimated population of over 229,080. Within the sub-city, there is a public hospital, a private hospital, nine public health centers, and one non-governmental organization health center.

Target Population

All Voluntary Counselling and Testing (VCT) clients, VCT program managers, and healthcare professionals offering VCT services at health facilities located in the Akaki Kality sub-city of Addis Ababa, Ethiopia constituted the target population for this study.

Source Population

The source population of this study consisted of the public health facilities in the Akaki Kality sub-city, as well as the program managers and VCT counsellors assigned to and operating within those facilities.

Study Population

Clients attending VCT services during the data collection period, program managers, VCT service providers, and Six months (months (November 2020 to April 2021) of VCT service-related documents in the selected health facilities were the study population of this evaluation.

Variables and Measurements of the Study

The process of implementing the Voluntary HIV/AIDS Counseling and testing service was evaluated using a formative evaluation approach. To determine the availability of program resources, 12 item indicators were used. Availability refers to the physical presence of services, supply of drugs, medical equipment, infrastructure, skilled health professionals, guidelines, and registers. The compliance domain was assessed using 12 items that measured healthcare providers’ adherence to national HIV/AIDS testing and counseling guidelines.¹³ To put it differently, accommodation pertains to how an organization is configured to cater to the convenience of its clients while also considering its own internal factors.^13,14 To assess client satisfaction, a set of 13-item questions was devised, with each question featuring a 5-point Likert scale that ranges from 1 (indicating great dissatisfaction) to 5 (indicating great satisfaction). The client satisfaction data was evaluated by converting the mean score percentage (%SM) of the scale. The application of this formula resulted in the generation of separate scores for each indicator. To ascertain the overall degree of contentment amongst the research participants, an average score was derived.¹⁵ Client satisfaction was judged using assessment metrics.

. During the assessment phase, the indicator weights were established by the stakeholders (Supplementary Table 1). These weights were computed using the formula: () The judgment parameter used was categorized as very good (>85), good (75–84.9%), fair (60–74.9%), and critical < 60% for each dimension and for the overall evaluation of the service.

Sample Size Determination and Sampling Procedure

The sample size for the survey was calculated using the single population proportion formula (, assuming a 95% level of confidence, 3% margin of error, 90% of clients were satisfied with the VCT service,¹⁶ A non-response rate of 10% was taken into consideration, resulting in a sampled population of 253 clients. Initially, five facilities were selected, using the lottery method.¹⁷ Participants were proportionally allocated to each of the selected health facilities based on their estimated three month client flows. Then participants were selected using systematic sampling technique.

The sample was drawn from VCT registers, service cards, and administrative records/reports spanning the period from November 2020 to April 2021. A total of 11 key informant interviews were conducted who were selected purposefully with expertise and knowledge to our study objective. The selection process involved intentionally choosing a focal person, the head of the facility at every health center, and the program coordinator from the sub-city’s HIV prevention and control office. The overall sample size was determined by the concept as per the data saturation. The study involved observing 25 client provider interaction. Furthermore, an assessment was done on the medical resources that were available at five different health centers.

Data Collection Tools and Procedures

Different tools were used to collect the necessary data. These included a checklist for an inventory of resources, a checklist for direct observations, reviewing documents, conducting key informant interviews using a semi-structured questionnaire, and exit interviews using structured questionnaires. The checklist that was employed to evaluate the availability of resources was adapted from HIV/AIDS testing guidelines.^8,18 To assess healthcare providers’ adherence to the national HIV testing protocol, the researchers employed both an observation checklist and a review of relevant documents. The national protocol for HIV testing closely aligns with the WHO guideline testing guidelines.^6,12,19 A structured questionnaire for exit interviews was adapted from national guidelines, literature, and the Joint United Nations Programme on HIV/AIDS (UNAIDS) evaluation guidelines.^12,19,20 The tools were initially designed in English, then translated to Amharic and Afan Oromo languages, and finally back-translated to English to ensure their consistency. Five healthcare providers and one supervisor were recruited for data collection. All of the data collectors held Bachelor of Science degrees in nursing, while the supervisor was a graduate of a public health with a Bachelor of Science degree. Prior to data collection, a training day was held to ensure that all data collectors were proficient in basic data collection techniques and client-handling. The qualitative part of the data was collected by the principal evaluator. A pre-test was conducted with 13 clients to assess questionnaire reliability, which demonstrated strong internal consistency with a Cronbach’s alpha value of 0.89. Throughout the data collection period, supervisors conducted daily checks to ensure the accuracy, consistency, and completeness of the collected data.

Data Management and Analysis

Quantitative data were checked for completeness, edited, coded, entered into Epi-data version 4.6 and exported into SPSS version 25 for analysis. Descriptive statistics were computed, and the results were presented using tables and narrations.

During key informant interviews, field notes were taken to capture important information and insights. Audio recordings of the interviews were also made and later transcribed into the local language before being translated into English. Thematic analysis was employed to analyze the qualitative data obtained from the study. The analysis focused on three key themes: the availability of resources, the compliance of service providers, and the satisfaction of clients with the VCT services. After analyzing the data, a set of predetermined judgemental criteria was used to assess the level of implementation for each dimension of the VCT service implementation process.

Result

Availability of Resources

The results showed that a total of 12 health care professionals were available in the five studied public health facilities of Akaki Kality sub-city to provide VCT service. All of them had got training on VCT. At least two healthcare professionals were assigned to each facility to provide the VCT services. The key informant interviewed indicated that there was enough staff to provide the service. However, there was a problem with the high turnover of trained staff and the resulting poorly implemented programmes.

We have no problem with the availability of staff since we tried to send more than one staff when the training opportunity comes. Actually, training opportunity was not given for the last six months since more emphasis was given to the emerging COVID-19 pandemic. [32 years old male, Head of the facility]

Although we had no issue with the availability of staff, high staff turnover was our main issue because until the newly assigned staff received training and became familiar with the service, it had an impact on the program’s proper implementation. [37 years old male, primary health care unit (PHCU) head]

Regarding the availability of basic materials and infrastructure needed for the program, all health facilities had separate waiting areas and counseling rooms for services with electric power, and only two health facilities had better audio-visual privacy. Three of the five facilities contained functional piped water.

We tried to make the room be on the last floor to enhance the audio-visual privacy, but it is still difficult to say that audio-visual privacy was maintained since most of the administrative offices are located on the last floor, and it is tiresome for some clients. [32 years old female PHCU head]

In all studied facilities, updated HIV testing and counseling guidelines and participant manuals were available. The 2007 guidelines were presented at only one facility.

I haven’t seen updated guidelines in this facility since I stayed in this area for more than a year. Mostly, I used the participant’s manuals soft copy [25 years old, VCT focal].

The problem with the availability of guidelines was raised from both professionals and the Ministry of Health. Occasionally, the healthcare professionals who are given copies of the guidelines fail to return them. Furthermore, both the Ministry of Health and the health bureau did not provide the updated guideline. [42 years old male, HIV prevention and control focal from the sub-city health administration]

All reporting and recording formats were available in the studied facilities. The problem with this approach is the utilization of a new format. Two of the facilities studied used an old format. Almost all facilities had protective materials, such as gloves, lancets, test kits, lab slips, client registration, IEC demonstration models, male and female condoms, and stationery, which were available and functional at the time of data collection (Table 1).

Different HMIS recording formats were supplied by the health bureau. We know that the HMIS recording format for this service was updated and used in some facilities, but we have not yet been able to obtain it. [27 years old female KII, focal person]

Table 1 Availability of Functional Necessary Resources to Provide VCT Service in Akaki Kality Sub-City Public Health Facilities, 2021

Another finding was the presence of a trained lab technician who supervised the counselor. In all facilities, no lab technician supervised the counselor. Testing and counseling services were provided by a single individual.

Presence of lab technicians to supervise the counselor is recommended by the guideline, but it is only applicable when the counselors are non-health professionals. In our case, the service is provided by health care workers who are trained in health and health-related fields. [26 years old male, VCT focal]

The overall availability score for VCT services was 84.8%, which was judged as good according to the pre-setted judgment criteria (Table 2).

Table 2 Summary of VCT Program Resource Availability Indicators in Akaki Kality Sub-City Public Health Facility June 2021

Compliance of VCT Service Providers to the Guideline

Record Review

During the past six months, a total of 2192 clients were recorded in the VCT register, and of these individuals, only 5.1% received positive test results. Of those having positive test results, 83.93% were linked to care and treatment centers (ART clinics). The remaining 16.07% were not linked to care and treatment centers. The key informant interviews supported the problem of linking all clients who had positive test results.

Sometimes known HIV-positive individuals who were already on ART and in the meantime discontinued the treatment and tried the traditional methods of healing for several months or years, visit the facility afterwards to find out their status and behave as if they were not aware of their status previously. If their test results show a positive outcome, they decline to be connected to the treatment and care service at that moment. [25 years old female, focal person]

Moreover, 12.4% of clients’ information was not included in the VCT register. Most respondents agreed that incomplete registration is common.

In our facility, most of the time incomplete registration of client information was not uncommon. Especially when new focal persons are assigned, they may miss completely registering the client’s information. We have faced such a problem while we conduct supervision with the facility supervision team. It may result from their negligence and poor monitoring and supervision as well. [37 years old male key informant facility head]

According to the six-month report and document review results, all the studied facilities sent complete reports to the sub-city health administration office. Additionally, three out of the five studied facilities received one supportive supervision from the town health bureau, but did not provide written feedback. The findings from key informants strengthen this result.

In the past six months, I received supervision from a supportive supervisor in the town health bureau and another in the sub-city health administration. However, these supervisions were not specifically focused on the VCT program. Instead, they were integrated with other services and often rushed, with no written feedback provided. [27 years old female, focal person]

Direct Observation of VCT Counselors in Providing the Service

From 20 direct observation sessions, all healthcare providers greeted their clients, explained and ensured confidentiality, actively listened to their clients, and discussed risk-reduction plans based on the guidelines (Table 3).

Table 3 Description of Provider-Patient Interaction and Communication on Services Provision of VCT in Akaki Kality Sub-City Public Health Facilities, 2021

All key informants agreed that confidentiality of clients were ensured.

The top priority in VCT service is the maintenance of confidentiality. Clients are not asked for their names, instead, we provide them with a code to ensure their privacy. This allows clients to feel at ease and discuss freely. [27 years old female KII, focal person]

Out of the 20 observation sessions conducted, it was noted that 15% of clients did not receive information about HIV tests as a part of the pre-test counseling procedure. This included details about the testing process, the significance of potential test results, and the window period, all of which are in accordance with the recommended guidelines (Table 4). This finding was supported by interviews with key informants.

Service providers often find it tedious and a waste of time, especially since a majority of the clients are urban residents who are already familiar with such matters. Due to this perception, some service providers may not give sufficient attention to providing information on these issues and may not strictly adhere to all of the guidelines presented. [36 years old male KII, head of the facility]

Table 4 Judgment Matrix for Compliance Dimension on Process Evaluation of VCT Service at Akaki Kality Sub-City Public Health Facilities June, 2021

The compliance dimension received an overall score of 85.38%. Furthermore, 100% of client records in the VCT register were accurately filled in. In 90% of observation sessions conducted, clients gave informed consent, while 100% of VCT clients were screened for STIs, which was one of the highest scores achieved. However, only 60% of health facilities that received supportive supervision with written feedback from the town health bureau in the last six months had ratings that were considered low under the compliance dimension (Table 4).

Client Satisfaction with VCT Service

The Societal, Demographic, and Economic Features of Clients

There were 244 clients who took part in the interviewer-administered questionnaire, resulting in a response rate of 96.4%. The average age of those who responded was 32.48 years, with a standard deviation of 9.6. A significant portion of the respondents, which was 30.3%, belonged to the age group of 25–30. Out of the participants, 45.9% were married while 37.7% were single. A majority of the participants (86.1%) were urban residents. In terms of education, 30.7% of respondents had completed diploma or higher education. Additionally, 27% of the participants or 66 individuals visited the facility based on recommendations from their friends. The majority of clients (62.7%) received their test results in less than 15 minutes. The duration of the counseling sessions ranged from 10 to 30 minutes, with a median duration of 15 minutes (Table 5).

Table 5 Socio-Demographic and General Characteristics of Respondents of Exit Interview for Process Evaluation of VCT Service in Public Health Facilities of Akaki Kality, June 2021 (n=244)

Clients Satisfaction

According to the results, 48.4% of the participants expressed satisfaction with how respectful the counselors were, while 8.2% said they were not satisfied. The majority of the clients (52.5%) reported being satisfied with the level of visual privacy provided in the counseling room, closely followed by those who were very satisfied (31.6%). Regarding the level of satisfaction with the opportunity provided by providers to participate in decisions concerning risk reduction, 42.2% were very satisfied, 41% were satisfied, and 7% were dissatisfied. Regarding comfort in the waiting area, 49.2% were satisfied, 34.8% were very satisfied, 7.8% were dissatisfied, and 0.8% were very dissatisfied. Moreover, most clients (49.6%) were satisfied with the clarity of the counselor’s explanation, and 20 (8.2%) were dissatisfied. Among the total respondents, 126 (51.6%) were satisfied with the overall service provision, and 15 (6.1%) were dissatisfied (Table 6).

Table 6 Level of Client Satisfaction on Each Satisfaction Measuring Items for Evaluation of VCT Service at Public Health Facilities of Akaki Kality Sub-City, Addis Ababa, Ethiopia, 2021

The level of client satisfaction was scored by transforming it into a percentage of the scale mean score for each satisfaction measuring item for the process evaluation of the VCT service. After the transformation, the data for each item became continuous, and the level of client satisfaction was determined based on the judgment parameter. Of all clients, 76.74% of them were satisfied with the respectfulness of service providers, 78.39% of them were satisfied with the physical privacy of the VCT clinic, and 76.95% of clients were satisfied with the clarity of the counselor’s explanation. Most clients were satisfied with the cleanliness of the counseling and testing rooms (80.5%). The mean overall client satisfaction score for the VCT service was 76.93% (Table 7).

Table 7 Judgment Matrix for Acceptability/Satisfaction Dimension on Evaluation of VCT Service at Akaki Kality Sub-City Public Health Facilities June, 2021

Discussion

VCT service of this study was assessed based on the availability, compliance, and satisfaction of clients. Resource inventory, observation of client-provider interaction, record review, key informant interview, and exit interview were conducted to determine the implementation status of VCT service.

Our evaluation revealed that the VCT service had sufficient resources such as trained personnel, IEC/BCC demonstration materials, gloves, lancets, tubes/needles/test tubes, all of which were available at 100% conformity to the minimum standards required by the national guidelines in the study facilities. However, test kits were not readily available in two out of the five facilities during the last three months due to stock-outs.¹² The stock-out was due to the absence of supply from NGOs since the COVID-19 pandemic and less emphasis was placed on the other cases. The present study’s findings contrast with that of a study conducted in Cape Town and Dares Salaam, which identified a significant issue with the shortage of trained personnel as a major obstacle to implementing the program in that particular context.^21,22 The observed inconsistency could be attributed to the variations in the study contexts. The setting in our case is solely in a rural area, which has better access to trained human power and equipment. This result also differs from a previous study conducted in Jimma town, which reported that only 87.5% of health facilities had trained counselors.²³ The observed inconsistency could potentially be explained by differences in both the time and context of the studies. In addition, it should be noted that all of the facilities examined in this study had designated spaces for counseling and testing, which is consistent with both the national HIV testing guidelines and WHO’s suggested protocols.^3,12 Nonetheless, this observation does not align with the results of a different study conducted in major health facilities in Jimma town which reported that 62.5% of health facilities lacked sufficient space to guarantee appropriate counseling.²³ This discrepancy could be attributed to variances in the study methodology and the sample sizes used.

Concerning the availability of the latest guidelines, there were no latest guidelines in any of the studied facilities, although the national guidelines recommend the presence of guidelines.^12,24 Our finding is lower than the one found in the previous study conducted in Malawi, South Africa, and Tanzania.¹⁸ This variation could be attributed to variations in healthcare systems and resource availability.

The results of the assessment indicated that all the facilities scrutinized had access to documentation and reporting methodologies in adherence to national regulations.^12,24 This conformity could be attributed to the ongoing health information revolution in the Ethiopian healthcare system. The presence of adequate resources might significantly facilitate the provision of services as expected. This could have a positive effect in achieving the first 95 of UNAIDS 95–95–95 fast tracking targets. Therefore, our results implied that there is still a need to assure the availability of resources in order to achieve the targets.

The results indicated that 85.38% of the VCT services followed the pre-test and post-test standards as well as receiving test results guidelines of national HIV testing and counselling. This consistency aligns with a study done in South Africa, which revealed that almost 98.6% of service delivery complied with national guidelines.²⁵

According to our assessment, 16.07% of clients did not adhere to the practice of connecting HIV-positive patients to proper treatment and care services. This result is not in line with national guideline recommendations.^12,24 This inconsistency may result from poor staff commitment and client behavior. The other finding from this evaluation showed that all clients underwent STI and TB screening and were registered appropriately, which is in line with the guideline recommendations.^12,24 In this evaluation, 80% of the clients’ confidentiality was ensured, which is not in line with the national guidelines that recommend ensuring the confidentiality of all clients.^12,24 The finding from this study has a higher figure than that of a research done in Kenya, which stated that privacy was maintained in only 37.4% of patients.²⁶ This discrepancy might be due to differences in the healthcare delivery systems. HIV testing and counseling services are confidential. Counselors should raise, among other issues, whom else the person may wish to inform and how they would like this to be done.⁸ Furthermore, lack of confidentiality may lead to low utilization of the service and may affect the achievement of the first target. Therefore, our findings imply that counselors should work on confidentiality of clients and providing the service in line with the standard guidelines.

The overall contentment of VCT clients satisfaction was 76.93%. This figure is less than that reported in Ethiopia, Addis Ababa, and South Africa, where client satisfaction levels were 89.9% and 90%, respectively.^16,25 This inconsistency could be attributed to variations in factors such as resource availability, sampling methods, and research design. The detected result is more significant than a preceding study carried out in India, where the cumulative satisfaction of clients was 60%.²⁷ The discrepancy could be due to variances in the socio-cultural backgrounds of the study population as well as differences in sample sizes.

In our study, clients expressed satisfaction rates for visual privacy of the counseling room (75.8%), waiting times to see counselors (75.2), and physical privacy of VCT clinics (78.39%). This result is comparable to that of a study conducted in Addis Ababa and the Adama Organization for Social Services for AIDS.²⁸

Strength and Limitation of the Evaluation

Our assessment utilizes three dimensions to evaluate VCT implementation processes, rendering it more reliable and inclusive of a broad range of implementation facets. Additionally, the use of both qualitative and quantitative methods (triangulation) aids us in garnering precise and comprehensive information.

The possible limitations might be that since this is a single case study design, it is not possible to generalize to other VCT sites. Counsellors may act differently when they are observed. The evaluation also recognized that social desirability bias, where clients may feel compelled to express gratitude towards their healthcare provider due to being in a health institution, could be a potential limitation.

Conclusion

In this evaluation, the implementation of the VCT service was judged good. Based on the judgment parameter, the availability of resources for providing VCT services at public health facilities of Akaki Kality sub-city was judged to be good, with the exception of some essential resource absence or shortage at health facilities, such as protected counseling and testing rooms, because even if all facilities had separate waiting and counseling rooms, they lacked audio-visual privacy. The compliance of service providers to the national guidelines was very good; however, utilizing IEC during counseling and linking all clients with positive test results to the treatment and care center was not practiced as recommended by the guidelines. Moreover, according to our evaluation, the level of overall client satisfaction with VCT services in this study was judged good.

Recommendation

Therefore, the Ministry of Health must provide updated guidelines and test kits in a timely manner. All types of training modalities should incorporate the provision of necessary equipment and supplies to enable effective and appropriate utilization of skills and education. The health bureau should design refreshment training for the staff and conduct program-based supportive supervision with written feedbackHaving separate rooms for testing and counseling in healthcare facilities would facilitate the maintenance of privacy, particularly audio and visual privacy during the VCT service. It is important to ensure that the client information in the register is complete, and attention should be given to this aspect. Healthcare providers should listen to clients with respect, allow them to respond, clarify their understanding of guidelines and explain how the clients can understand and use them.