Level of Implementation Fidelity of Youth-Friendly Services in Northwest Ethiopia

Background

Youth are people aged 15 to 24.¹ Globally, there are around 1.8 billion youth, accounting for one out of six people. Of them, 87% live in developing countries.^2–4 In the Ethiopian context, youth represent about 33% of the national population dividend.⁵ They have faced many health problems that could be due to low Youth-Friendly Service (YFS) utilization, poor YFS delivery, cultural and other social determinants.^6–8

YFS is a promising evidence-based intervention, which was designed by the World Health Organization (WHO) that aimed to avert the Sexual and Reproductive Health (SRH) problems of youth.^9,10 The WHO recommended YFS should be accessible, acceptable, equitable, appropriate and effective for youth^11–14 and it has previously been highlighted as a successful model for providing SRH services integrated within a public health system.^15,16 A range of services like counseling on sexuality, Human Immune Deficient Virus (HIV) counseling and testing, pregnancy testing, contraceptive provision, Sexually Transmitted Infections (STIs) screening and management, abortion care, and other medical services are integrated and provided in the YFS.^17,18 In Ethiopia, YFS has been implemented in public health centers with monitoring and supervision held by Federal Ministry of Health (FMoH), since 2013.¹⁶

Currently, many youths require SRH services that can address their diverse needs.^19–21 Hence, YFS was developed as a promising approach to deliver health services to meet youth’s SRH need.^9,10,17 Even if YFS intervention is implemented in countries like Ethiopia,¹⁶ youth suffer from many SRH problems like teenage pregnancy, fistula, abortion, and STIs including HIV.^{7,15,19,20,22} Low implementation fidelity of YFS could be one of the possible reasons claimed for the high prevalence of SRH problems among youth.^23,24

Most previous studies focused on the prevalence of YFS utilization,^25–27 barriers limited youth’s access to YFS,^28,29 and few on YFS quality of delivery.³⁰ Besides, low quality of delivery^30,31 and low positive experience among youth with the service³² were some of the reasons claimed for the low proportion of YFS use. Pieces of evidence showed, implementing programs with high fidelity, play a vital role in increasing client satisfaction, service utilization and to achieve the intended intervention outcomes.^23,24,33 Though various international and national organizations have assisted in the expansion of YFS, there is study limitation on its level of implementation fidelity.¹⁵ Complex programs, such as YFS, in which many services are integrated and provided at a time, have faced challenges in delivering the intervention with high implementation fidelity.¹⁸

Monitoring implementation fidelity contributes to future improvements of interventions, which are necessary to increase the likelihood of reaching intended outcomes.^{23,24,34–36} Furthermore, studying on Fidelity of Implementation (FoI) efforts may also lead to better replications and sustainability of an intervention like YFS.^35,37 Therefore, it is vital to investigate the level of implementation fidelity of YFS.³²

Measuring the implementation fidelity of services like YFS are conceptually developed from three major constructs called adherence, quality of service delivery and participant responsiveness.^23,24 Hence, it is vital to quantify the three main constructs that are intended to measure fidelity of YFS.^23,24

There is an evidence gap on the level of implementation fidelity of YFS in developing countries including Ethiopia, specifically in the study area. The main objective of this study was to describe the level of implementation fidelity of YFS in this specific study area. To measure the overall fidelity of YFS, the three constructs used to measure the fidelity of YFS (adherence, quality of YFS delivery and participant responsiveness) were quantified separately. Therefore, this study aimed to investigate the level of implementation fidelity of YFS in public health centers in Central Gondar Zone, northwest Ethiopia. The findings of the study may help to strengthen and implement the YFS with high fidelity.

Methods

Study Design and Setting

An institution-based cross-sectional study was conducted among youth who utilized the YFS, from randomly selected public health centers (implementing the intervention), in Central Gondar Zone, northwest Ethiopia, from September to December 2019. Central Gondar Zone has a total of 15 districts, 466 kebeles (the smallest administrative units), and a total population of 2,655,844, while youth accounting for 34.6% of the total population. There are 84 public health centers, though 35 of the HCs were implementing YFS, according to the 2018 Zonal report.³⁸

Study Population and Inclusion Criteria

The source population of the study was youth of age 15–24 who came for YFS in the study area and the study population were youth of age 15–24 who utilized YFS in that specific area and available during at the time of the study. Youth who are available provided informed and written consent during the data collection period were included in the study; while those youth who were critically ill and unable to respond during the data collection period.

Sample-Size Determination

To determine the sample size for this study, a pilot study was conducted from 60 youths attending YFS, from randomly selected HCs called Enfranz, Han and Bahir Dar. These three HCs were selected as they have already implementing YFS and have a similar setting with the main study area. Then, the sample size was calculated by using the single population proportion formula³⁹ and taking the proportion of the high level of implementation fidelity of YFS P=26.7% from the pilot study, 95% CI, margin of error of 4%, design effect of 2, and 10% non-response rate. The final calculated sample size was 1034.

Sampling Procedure

From the 15 districts in the zone, 6 were randomly selected by lottery method. Then, if there are two or more HCs implementing YFS in each district, 1 HC, was selected by lottery method. Finally, 11 HCs named Gondar (GHC), Azezo (AZHC), Tseda (THC), Gebriel (GEBHC), Woleka (WHC), Maraki (MHC), Amba Giorgis (AGHC), Maksegnit (MKHC), Kolladiba (KHC), Chuahit (CHHC) and Delgie (DHC) were included. Then, samples were correspondingly allocated across each HC to get representative participants. Thereafter, participants were selected using a systematic sampling technique during working hours of the week in each HC.

Tool Development, Validation and Adaptation Processes Used to Measure the Implementation Fidelity of YFS

An instrument comprised 17 general items; which measured the socio-demographic and other individual characteristics were used. Also, a validated scale with 65 items with 5 category Likert scale scores for assessing the implementation fidelity of YFS was employed. The 65 items scales were mainly from the WHO-Plus standard tool (quality assessment tool).⁸ The scale was developed to measure the three dimensions of fidelity ie, adherence, quality of service delivery, and participant responsiveness. Then, a fidelity score was developed for each fidelity domain, based on 5 Likert scale level. The scale passed the standard tool validation process, starting from face validity, content validity, construct validity, pilot tested and finally internal consistency was checked using the information from the pilot study. The overall instrument development, validation and adaptation processes are described below.

Item Generation

A comprehensive literature review was done and a pool of items having a total of 113 questions intended to measure the three constructs (adherence, quality of delivery and participant responsiveness) of implementation fidelity of YFS was generated as an initial point.

Face and Content Validity

The first round of face and content validity was determined by 6 professionals’ who independently reviewed items in the questionnaire and provided ratings on the importance of each item. In addition, they rated their level of agreement for the importance of each question as very important, important, somewhat important or not important. Moreover, they qualitatively evaluated/checked on layout and readability, meaning and clarity of wording, feasibility of each item related to study participants.

In the second round of face and content validity, a panned discussion including another 6 professional experts and 2 language experts was conducted. Hence, corrections like wordings, questions were phrased in positive tone, misclassification of variables, logical order of items from all constructs, repeated questions, ambiguous and vague words and double question at a time were separated. In addition, logical arrangement, appearance, layout, acronym, jargons and technical language removed, leading questions avoided, and many editorial issues were raised and edited. Finally, the comments, corrections, suggestions, judgments and opinions of the professional and language experts were considered.

Standard translation/back-translation procedures were performed to prepare a preliminary draft of the scale in Amharic (local language). In the first stage, the original instrument was translated into Amharic by two professional language translators; and in the next stage, the Amharic version was translated into English by two other professional translators who were familiar with the main study topic. In the third stage, the back translated English version of the tool was compared with the original version in terms of content, and idiomatic and semantic affinity. Minor corrections were made as a consequence and the final draft was approved by the researchers.

Item Reduction

After the first and second rounds of expert validity, 42 items were reduced and a total of 71 items were left for the pilot study. These 42 items were reduced due to criteria like decisions by experts as not content valid (10 items) and duplicates (32 items). Then, the pilot study was conducted using the 71 items to cheek the cultural adaptation of the tool.

Results of Internal Consistency After Conducting the Pilot Study

After the pilot study, 6 items were reduced after checking for internal consistency using the Crombach alpha value. Finally a valid and reliable instrument having 65 items in the three constructs were developed to measure the FoI of YFS.

Finally, 9 items were developed to measure the participant responsiveness dimension, with high internal consistency (Cronbach’s alpha value of 0.85), 15 items were developed to measure the adherence dimension with Cronbach’s alpha value of 0.91 and 41 items were developed to measure the quality of delivery dimension, with high internal consistency having Cronbach’s alpha value of 0.93. The quality of delivery dimension was further constructed based on the Donavidian model quality of care framework, that aimed to assess the structural quality (11 items), process quality (23 items) and outcome quality dimensions (7 items).

Data Collection Procedures

Data were collected by an interviewer-administered, predetermined and validated questionnaire. Eleven BSc holders (5 Health Officers, 6 Midwives), who had special training on YFS and working out of the data collection area, collected the data. The data collectors were health care professionals who were working as heath care providers out of the study area. In addition, the data collectors were not involved in the implementation of the YFS intervention in any of the settings. One supervisor having a Master of public health and with work experience on supervision in research data collection was involved.

Data Quality Control

To control the data quality, three days of training was provided to 11 data collectors and one supervisor before the actual data collection. Besides, the instrument was validated, so that the face validity, content validity, construct validity, internal consistency of the instrument was found high. Moreover, appropriate modifications were made on the instrument, after conducting the pilot study. The questionnaire was translated into local language (Amharic) and back-translated to the English by one language expert and one public health expert. Then, to ensure consistency, the Amharic version of the instrument was back-translated into the English language by another English language expert and by another professional expert.

Keeping Confidentiality

The names and other identifiers of the respondents were not recorded in the questionnaire. The information collected in each health facility was submitted to the supervisor and the principal investigator on a daily bases. The collected data were locked in a secured shelf and nobody will be allowed to see the information (except the research team).

Data Analysis

Data were analyzed using STATA version 14. Descriptive statistics with summaries of each fidelity dimension was computed first; afterwards the overall fidelity score was computed using 3 main dimensions of fidelity. The unit of analysis was at an individual level, and then aggregated into the HC level with total implementation fidelity scores were computed separately into three levels. Those youth whose total fidelity score of greater than or equal to 75% (≥243.75/325) was declared as had received the YFS intervention with high Fidelity of Implementation (FoI), those youth whose total fidelity score from 50% (162.5/325) to 75% (243.74/325) was declared as had received the YFS intervention with medium FoI and those youth whose total fidelity score below 50% (<162.5/325) were declared as had received the YFS intervention with low FoI.^8,36 Regarding each dimension of fidelity measure, we used similar cutoff percentages for each dimension and the results are presented in tables.

Results

Socio-Demographic Characteristics

Of the total 1034 youths, 1029 (99.5%) responded to the survey. The majority, 717 (69.7%), were between age 20–24, and 752 (73.1%) were females. Regarding their religion, 874 (84.8%), were Orthodox Christians, and 781 (75.9%) of them were urban residents. Concerning their educational status, the majority, 453 (44.0) of them were attending secondary education, and 601 (58.4%) were unmarried (Table 1).

Table 1 Socio-Demographic Characteristics of the Respondents, Northwest Ethiopia in 2019

Youth-Friendly Services Utilization

Regarding the YFS services, 223 (21.7%) got contraceptive counseling, 308 (29.9%) got contraceptive method collection, 263 (25.6%) got screening for STIs, 167 (16.2) peer counseling services, 49 (6.5%) abortion services, 38 (5.0%) post-abortion care services, 173 (23.0%) pregnancy testing services, and 75 (7.3%) got counseling on the risks of having sex before 18 years (Table 2).

Table 2 Youth-Friendly Services Utilization Related Characteristics of the Respondents in Northwest Ethiopia, 2019

Level of Adherence to Youth-Friendly Services

Considering the level of adherence dimension, 190 (18.5%) youth get the YFS with a low level of adherence, 598 (58.1%) with a medium and 241 (23.4%) with a high level of adherence. Tseda HC provided the YFS with the highest level of adherence 55 (58.5%), followed by Delgie HC, 53 (56.4%) (Table 3).

Table 3 Level of Adherence to YFS Across Health Centers in Northwest Ethiopia, 2019

Level of Quality of Delivery of Youth-Friendly Services

Regarding the structural quality of care dimension, 127 (12.3%) get the YFS with a low level of the structural quality of care, 690 (67.1%) get the YFS with a medium, and 212 (20.6%), get the YFS with a high level of the structural quality of care. Azezo HC provided the YFS with a high level of the structural quality of care 45 (47.9%) followed by Tseda HC 39 (41.5%). On the process quality of care dimension, 103 (10.0%), get the YFS with a low level of process quality of care, 603 (58.6%), get the YFS with a medium, and 323 (31.4%), get the YFS with a high level of process quality of care. Azezo HC provided the YFS with a high level of process quality of care 82 (87.2%) followed by Tseda HC 63 (67.0%). Considering the outcome quality of care, 146 (14.2%) get the YFS with a low level of outcome quality of care, 375 (36.4%) get the YFS with a medium, and 508 (49.4%) get the YFS with a high level of outcome quality of care. Azezo HC provided the YFS with a high level of outcome quality 82 (87.2%) followed by Delgie HC 70 (74.5%) (Table 4).

Table 4 Level of Structural, Process and Outcome Quality of YFS Across Health Centres in Northwest Ethiopia, 2019

Level of Participant Responsiveness on Youth-Friendly Services

Regarding the participant responsiveness fidelity dimension, 613 (59.6%) get the YFS with a low level of participant responsiveness, 302 (29.3%) get the YFS with a medium, and 114 (11.1%), get the YFS with a high level of participant responsiveness. Tseda HC has the highest level of participant responsiveness 43 (45.7%) followed by Kolladiba HC 28 (29.8%) (Table 5).

Table 5 Level of Participant Responsiveness of YFS Across Health Centers in Northwest Ethiopia, 2019

The Overall Level of Fidelity of Implementation of Youth-Friendly Services

The overall fidelity measure showed that only 48 (4.7%; 1.6–11.3%) of youth received the YFS with a high level of implementation fidelity, followed by 770 (74.8%, 57.6–90.5%) with a medium and 211 (20.5%; 12.7–31.0%) with a low level of implementation fidelity. Tseda HC provided the YFS with a relatively higher level of FoI 19 (20.2%), followed by Azezo HC 17 (18.1%) (Table 6).

Table 6 The Overall Level of Fidelity of Implementation of YFS Across Health Centers in Northwest Ethiopia, 2019

Discussion

This study assessed the level of implementation fidelity of YFS among youth attending YFS services in 11 public health centers found in Central Gondar Zone. The overall implementation fidelity of YFS was found to be low as compared to the WHO recommended implementation standard of YFS delivery. All the health centers have scored below the standard level of FoI of YFS.

The results showed that only 48 (4.7%) of the respondents got the YFS with a high level of implementation fidelity. Besides, 770 (74.8%) get the YFS with a medium, and 211 (20.5%) get the YFS with a low level of fidelity of implementation.

This finding is lower than the WHO standard⁸ and the National Adolescent and Youth Health Strategy of Ethiopia,¹⁵ whereby the standards mentioned as the YFS should be delivered with a high level of implementation fidelity and quality. The possible explanations for the lower level of Fidelity of Implementation of YFS in the current study could be the nature of the YFS intervention, as it is a complex intervention and resource-intensive demanding trained providers and many other resources. Hence, resource limitation, lack of training of the YFS providers’, budget constraint, and lack of NGOs support in the study area could be the potential reasons.⁴⁰ This finding is also lower than a study conducted in Spain, where 57.2% of youth got the YFS with a high level of FoI.³⁶ The possible reason for the low level of FoI of YFS in the current study as compared to the aforementioned study could be the difference in the health system infrastructure between the two countries. As Spain is a developed nation as compared to Ethiopia, the health care system including the YFS program/provision may be stronger as compared to Ethiopia. The other possible justification could be the presence of some methodological differences in the dimensions used to measure FoI of YFS in the two studies. In the study conducted in Spain, the authors used dose, adherence and acceptance dimensions as a measure of fidelity,³⁶ while in the current study adherence, quality of YFS care and participant responsiveness were used. Usually, the quality of care dimension is the very difficult part to be delivered with high fidelity, and hence the overall fidelity measure in the current study is lower.⁸

Regarding the adherence dimension, only 18.4% of youth get the YFS with a high level of adherence. This finding is lower than the WHO⁸ and the National Adolescent and Youth Health Strategy of Ethiopia.¹⁵ The possible explanation could be there could be a lack of commitment of the YFS providers in providing the YFS with a high level of adherence. Also, lack of training at all or lack of refresher training on YFS could be another possible reason for the provision of YFS with a low level of adherence. Usually, training motivates providers to deliver YFS with high adherence or fidelity.⁴¹ Moreover, many of the HCs may not have the YFS national standard manual in their clinic so that they may not read it and hence lack of adherence happens. Furthermore, lack of supportive supervision from the district, Zonal and Regional health offices may also be another justification for the low level of adherence in YFS provision.

The level of adherence is also lower as compared to a previous study conducted in Spain, where 30.4% of youth got the YFS with 100% adherence and 69.6% of the respondents got the YFS between 70% and 90% level of adherence.³⁶ The possible justification could be the presence of disparities in the YFS setups including the presence and quality of training provided for health care providers in YFS provision between the countries.

Regarding the YFS quality of delivery fidelity dimension, only 12.3% in the structural dimension, 10% in the process dimension, and 14.2% in the outcome dimension get the services with a high level of quality. All of the YFS sites scored lower than the WHO⁸ and the National Adolescent and Youth Health Strategy¹⁵ standard (75%) in the three of quality dimensions and the overall quality of YFS is categorized as “not good quality” or “below the standard.” Besides, the quality of delivery score in this study is lower than a study conducted in Arbaminch; whereby the overall quality of YFS delivery score input, process, and youth clients’ satisfaction was 54.41%, 42.0%, and 49.1%, respectively.³⁰ The possible reason for the lower findings of the current study as compared to a study conducted in Arbamnich could be the presence of variation in the study settings between the two studies conducted. The aforementioned study was conducted only in urban settings whereas in the current study both urban and rural HCs were considered.³⁰ Usually, in the urban settings, the YFS clinic gets better support from many stakeholders, NGOs and even the Regional and Zonal health offices. Likewise, in urban HCs there are more health care providers who are trained and experienced in providing YFS as compared to rural HCs.¹⁰

Furthermore, the YFS quality of delivery measure is still lower as compared to a study conducted in Mexico (17.5%).⁴² The possible explanation for the lower quality score in the current study as compared to Mexico’s study could be the presence of difference in the socio-economic, the overall health care system and the YFS program competency between the two countries.^16,42

The participant responsiveness dimension of this study showed that only 59.5% of youth got the YFS with a high level of fidelity. This finding is against the WHO standard and the National Adolescent and Youth Health Strategy of Ethiopia, whereby full youth participation and involvement in the YFS delivery is a milestone in YFS delivery.^8,15 The possible justification for the low level of participant responsiveness in the current study could be, since the study included rural HCs, in most of the HCs youth may not participate in the design, planning and as a peer counselor in the YFS provision. Further, there could be poor youth and community perception for YFS in the study area that may lower youth involvement in YFS. This finding implies that participating youth in the YFS program design and service provision is crucial to deliver the YFS with high fidelity. Therefore, programmers, implementers, strategists and policymakers better consider youth involvement in the YFS program. Further research has to be conducted to explore the barriers to the implementation fidelity of youth-friendly services.

The findings of the study implied that rigorous efforts to build systems that apply the policy and principles of implementation fidelity of YFS in the health facilities are vital. These initiatives will not only benefit youth but the health system overall, as the principles for implementation fidelity of the National Adolescent and Youth Health Strategy are in step with those of YFS to be delivered with a high level of adherence, quality of delivery and youth participation.¹⁵ Another implication of the study finding is considering high levels of implementation fidelity of YFS is compulsory to avert the SRH problems among youth, in addition to the YFS service expansion. Furthermore, investing in effective interventions (like YFS), is important to improve its implementation fidelity. There is a need to strengthen the YFS to be delivered with a high level of fidelity to achieve the desired intervention outcomes within a short period of time.

Limitations

As a limitation of this study, the measure of Fidelity of implementation of YFS was from youth perspectives, that means, the study did not consider the providers’ perspective and direct observation. Also, the study did not explore barriers to the implementation fidelity of YFS qualitatively.

Conclusion

In this study, the level of implementation fidelity of YFS is found to be low in all three dimensions of fidelity. The findings showed a discrepancy between the implementation policy and the actual/practical health facility level performance. Previous studies only investigated the proportion of YFS service utilization among youth, without considering its implementation fidelity. Hence, this study generates new evidence by showing the actual program-level performance on the implementation fidelity of YFS which has a better input to strengthen the YFS intervention.

Rigorous efforts to build systems that apply the policy and principles of implementation fidelity of YFS in the health facilities are recommended. Therefore, the government, partners, programmers and implementers shall give primer attention on improving the implementation fidelity of YFS. Further studies that can address the barriers to the implementation fidelity of Youth-Friendly Services are recommended. Finally, the authors’ recommend to conduct studies that will identify the factors that contribute to the fidelity of YFS by employing advanced statistical regression models.