Perceptions on Data Quality, Use, and Management Following the Adoption of Tablet-Based Electronic Health Records: Results from a Pre–Post Survey with District Health Officers in Ghana

Introduction

An Electronic Health Record (EHR) is defined as a computerized health record system that can be used for data capture and storage at a health facility.¹ It has been believed that EHR has advantages over paper-based systems in terms of timely and efficient data sharing, management, and analysis. Generally, EHR can play a key role in solving problems associated with paper-based communication. For example, paper-based health records have disadvantages such as data access being limited to a single user at a time, the requirement for large physical storage space, inefficient data transfer, risk of data loss, and difficulty in longitudinal record management, which can all be tackled by a well-functioning EHR.² Also, it has been documented that EHR has the potential to improve data quality and the reporting process.³

For these advantages, EHR has been recognized as a key tool for strengthening health information systems in low- and middle-income countries (LMICs). Over the past decade, considerable effort has been devoted to the implementation of EHR in several LMICs, as evidenced by previous literature.⁴ Largely triggered by vertical health programs such as HIV/AIDS treatment or maternal and child health (MCH) interventions, EHR projects were launched in countries such as Zambia, Ethiopia, Kenya, Uganda, Malawi, and Ghana.^5–10 Additionally, open-source EHR systems, including District Health Information Software 2 (DHIS2) or Open Medical Record System (OpenMRS), paved the way for affordable EHR systems in low-resource settings.^3,11 In an effort to promote the uptake of EHR in LMICs, the World Health Organization (WHO) published a guideline entitled “Electronic Health Records: Manual for Developing Countries” over a decade ago.¹²

Ghana is one of the adopters of DHIS2, whose implementation was led by the Ghana Health Service (GHS).⁸ During the early stages of DHIS2 implementation, the process was still semi-electronic. In 2007, the DHIS2-based District Health Information Management System (DHIMS) was initiated by the GHS, but it only allowed aggregate data management at the district level. At the facility level, individual data was captured on a paper-based system and the facility staff submitted a monthly report by manually collating the data from the registers. Recently, however, a mobile application called e-Tracker has been rolled out to community health facilities in Ghana, which enables individual-level data collection on a tablet computer. By using the e-Tracker, frontline health workers at Community-based Health and Planning Services (CHPS) compounds can enter, track, and manage the data electronically. This electronic data collected from the facilities will be synced and reported at the district level for decision-making.

Despite prolonged efforts for EHR implementation in LMICs including Ghana, research gaps still exist. First, participants of the published studies are mostly frontline health workers, while little is known about the experiences and perceptions of policy-makers or health information managers. A recent systematic review on EHR pointed out that the majority of study participants were clinicians rather than non-clinicians such as administrative staff, IT staff, or managers.¹³ Similarly, most review studies on mobile health (mHealth) for maternal health in LMICs focused on healthcare workers in clinical settings and rarely expanded the scope of their target population.^14,15 Although there is one study that discussed the opinions of health managers and policy-makers along with the experiences of frontline health staff, it was a qualitative study involving only 14 participants.¹⁶ Therefore, there is a lack of evidence on how point-of-care data collected through EHR can be effectively translated into a policy or quality improvement.

Second, most studies on EHR in the context of LMICs report cross-sectional results or the technical process of EHR development rather than pre–post evaluation of EHR implementation. For example, previous literature discussed results from the assessment of the EHR system at a single point in time.^9,17,18 Also, other studies focused on the design and development of EHR, providing descriptions about EHR implementation instead of empirical evidence involving a follow-up.^19–21 There is another mixed-method study on e-Tracker supported by the United States Agency for International Development (USAID), but it involved a cross-sectional interview with a small number of district-level health information officers.²² Even though these studies may give insights into preliminary steps for improving workflow, more information would be needed to achieve sustainable and long-term change.

To address these research gaps, this study aimed to assess the perceptions of district health officers on e-Tracker roll out across two regions of Ghana using a pre- and post-survey design. Specifically, it seeks to examine whether the e-Tracker would improve the data quality, promote the usage of data for action, and enhance efficiency in data management from the perspectives of district health officers. In doing so, the study seeks to generate evidence for designing strategies to maximize the benefits of the new EHR system.

Materials and Methods

Study Design

A pre- and post-survey for district health officers including district health information officers (DHIO) and district public health nurses (DPHN) was conducted using a self-administered questionnaire developed by the research team. The survey was designed to assess the perceptions of district health officers in terms of MCH data quality, use, and management before and after the adoption of e-Tracker by frontline health staff such as community health nurses (CHNs) and community health officers (CHOs). The quality, use, and management of MCH data was investigated because the e-Tracker was initially introduced to allow the data entry for maternal or child health issues.

In order to incorporate the priority and actual practice of district health officers, we attempted to hear from them during the survey development process. Instead of borrowing common definitions of data quality, usage, and efficiency from health data management literature, the question items were designed to ask key aspects of each domain from the perspectives of district health officers. To that end, the research team had several meetings with the district health officers and CHNs to examine their workflow and expected challenges. Based on these meetings, additional field visits, and a literature review, a pilot version of the questionnaire was revised accordingly.^7,8,23–25 The survey was developed in English and pilot-tested with four district health officers in Volta region before the e-Tracker roll out. Starting from November 2018, the pre-survey was administered during the initial e-Tracker training sessions in Volta and Eastern regions. During the refresher training session, the same set of questions were then asked again as part of the post-survey for the participants of the pre-survey. The post-surveys for both regions were completed in November 2019. The training sessions on data management, quality, and use were led by the Ghana Health Service–Ministry of Health and were supported by multiple stakeholders including the research team. Examples of topics covered in training sessions include generating a report or checking data completeness.

Study Participants and Sampling

Volta (recently divided into Oti and Volta regions) and Eastern regions were chosen as the target of this study, because they implemented the e-Tracker during early stages of the planned national roll-out. In Volta and Eastern regions, there are 25 and 26 districts, respectively, and one or two DHIOs or DPHNs are responsible for each district. The DHIO or DPHNs from three or four districts participated in each e-Tracker training session together. Although we used non-probability, a purposive sampling method by using the e-Tracker training sessions as a venue for the survey administration, the sample was largely representative of Volta and Eastern regions. To be specific, the district-level health officers from 24 out of 25 districts in Volta and 24 out of 26 districts in Eastern regions participated in both pre- and post-surveys. Forty DHIOs or DPHNs from Volta and 33 DHIOs or DPHNs from Eastern region participated in both pre- and post-survey, resulting in 73 participants in total.

Measurements

Sociodemographic information as well as the internet connection environment at the district health office was measured. To measure the changes in perceptions on data quality, use, and management, a set of questions were asked accordingly. First, questions regarding perceptions on data quality include data consistency across community health facilities, reliability, timeliness, completeness, and accuracy. These questions were asked to investigate the changes in district-level health officers’ perceptions on data quality before and after the adoption of e-tracker. In addition, the survey participants were asked if there is any important MCH data that has not been collected from the community health facilities. These questions for data quality were measured on a 5-point Likert scale.

Second, the data use was measured by asking DHIOs and DPHNs if they used MCH data compiled from the CHPS compound to carry out several tasks. These tasks include designing maternal, child health, or family planning interventions, requesting vaccines for the district, providing guidance for the CHPS compound, evaluating performance of CHNs, or writing a research proposal on MCH. Given that the DHIOs and DPHNs had already been managing the aggregate data electronically at the district level, the introduction of e-Tracker was expected to facilitate the use of MCH data for these tasks even further. The questions were designed in binary yes–no type, and the respondents were given an additional answer choice of “not applicable” where the data was not available for the specific task.

Third, the data management efficiency was measured by examining any delays in monthly MCH report from the CHPS compound, hours spent for validating the MCH data reported by the CHPS compound, and minutes spent for generating aggregate data. Also, the respondents were asked how many days they worked beyond working hours during the past month due to MCH data management. For these questions, the respondents were asked to provide numeric values in open-ended format.

Statistical Analysis

A paired dataset for 73 respondents from districts across Volta and Eastern regions was prepared because the pre- and post-survey data should be matched for comparison. These two regions that implemented the e-Tracker during early stages have 51 districts in total, where one or two DHIOs or DPHNs work for each district. Therefore, the entire population size of our sample did not exceed 100. Given the inherently small sample size, appropriate statistical tests were performed to investigate any changes on perceptions between the pre- and post-survey.

First, McNemar’s test was used to identify statistically significant differences in perceptions on data quality, use, and stability of internet connection before and after the implementation of the e-Tracker. McNemar’s test can be used to compare the marginal frequencies of two binary outcomes from a matched pair.^26,27 On the one hand, the questions for data quality and internet connection environment which were measured on a 5-point Likert scale were transformed into a yes–no binary outcome. The scales of 4 and 5 were recoded as “yes” and the scales of 1, 2, and 3 were recoded as “no”. On the other hand, the questions for data use were measured by binary variables, because the respondents were asked whether they used the data for a certain task or not.

Second, Wilcoxon Signed-Rank test was performed to compare the pre- and post-survey results on efficiency in MCH data management practice among district health officers. Wilcoxon Signed-Rank test is an analysis method for a matched pair and can be applied to compare differences for continuous variables that are not necessarily normally distributed.²⁷ The questions on data management practice consist of open-ended questions to indicate numeric values such as the average percentage of delayed monthly report or average time spent on MCH data validation.

Third, Chi-square test and Mann–Whitney U-test were performed to identify any statistically significant differences in demographic characteristics among study participants from Volta and Eastern regions. The Chi-square test was used to investigate statistical differences for categorical variables including sex, education level, and position. The Mann–Whitney U-test rather than the t-test was applied for the age variable because it did not show a normal distribution (Shapiro–Wilk test p-value<0.001).

Ethics

Ethical approval for this study was obtained from the Ghana Health Service Ethics Review Committee (GHS-ERC Number: GHS-ERC009/09/18). Participants of the study were given an information sheet regarding the research and signed the informed consent form.

Results

Sociodemographic Characteristics and Perceptions on Internet Connection

A total of 73 respondents completed both pre- and post-surveys in Volta and Eastern regions. Overall, there were no statistically significant differences between the two regions in terms of sociodemographic characteristics and their perceptions on the stability of internet connection. In Volta and Eastern regions, 52% and 57% of the respondents were male, respectively (Chi-square p-value=0.665). With regard to the education level, 50% had a diploma and 45% had a bachelor’s degree in Volta region. Similarly, those who had a diploma and bachelor’s degree were 48% each in Eastern region. The majority of respondents from both regions were DHIOs, accounting for 52% in Volta and 70% in Eastern regions (Chi-square p-value=0.255). The mean ages for Volta and Eastern regions were 39.6 (SD=9.5) and 37.7 (SD=9.0), respectively (Mann Whitney U-test p-value=0.559). These results imply that demographic characteristics of the district health officers from the two regions are not too different from each other. The percentages of district health officers who changed their views on the stability of internet connection were 9% (pre: no, post: yes) and 8% (pre: yes, post: no), respectively, and no statistically significant difference was found (McNemar’s test p-value=0.763). However, less than 50% of the respondents reported that they had stable internet connection at both pre- and post-surveys. The results for the comparison of sociodemographic characteristics and perceptions on internet connection are presented in Table 1.

Table 1 Comparison of Demographic Characteristics (Volta and Eastern Regions) and Internet Connection Environment (Pre and Post)

Data Quality

Table 2 illustrates the pre- and post-survey results on data quality. From the paired cases comparison, the percentages of respondents who changed their perceptions on the variability in MCH data accuracy across CHPS compound were similar, at 24% (pre: no, post: yes) and 25% (pre: yes, post: no), respectively (p-value=0.866). No statistically significant differences were found for the other aspects of data quality as well, with over 60% of the respondents remaining positive towards reliability (62%), timeliness (80%), completeness (67%) and accuracy (60%) of the MCH data reported from the CHPS compound. Also, there was no statistically significant difference between pre- and post-survey proportions of those who said that important MCH data are not collected at the CHPS level (p-value=0.197).

Table 2 Comparison of Pre- and Post-Survey Results on Data Quality

Data Use for Relevant Tasks

Table 3 shows the comparison of pre- and post-survey results on data use for various tasks over the past month. The tasks included the design of MCH or family planning interventions, request for the vaccines, guidance and evaluation for CHPS compound staff, and writing of a research proposal. A statistically significant decrease was found between pre- and post-survey in the use of MCH data for requesting vaccines (p-value=0.035). There were no statistically significant differences for the other tasks, with over 70% of respondents remaining engaging in guidance (77%) and evaluation (73%) for CHPS compound staff before and after the implementation of the e-Tracker.

Table 3 Comparison of Pre- and Post-Survey Results on Data Use for Various Tasks Over the Past Month

Efficiency in Data Management

Unlike the domains for data quality and use that did not demonstrate a statistically significant improvement in pre- and post-surveys, several aspects of efficiency in data management showed statistically significant improvement. As shown in Table 4, there was a statistically significant improvement with decreases in time spent on certain tasks. Specifically, the percentages of respondents who have experienced a decrease in hours spent on MCH data validation and minutes spent on generating aggregate data for vaccination coverage and antenatal care coverage were 54.5%, 50%, and 53.2%, respectively (p-values=0.027; 0.05; 0.047). There were no statistically significant differences between pre- and post-surveys in terms of percentages of delayed monthly report and days overworked due to MCH data management (p-values=0.972; 0.967).

Table 4 Comparison of Pre- and Post-Survey Results on Efficiency in Data Management Practice Over the Past Month

Discussion

Perceptions on Data Quality, Use, and Management

By conducting a pre- and post-survey in two regions of Ghana, this study aimed to evaluate how district health officers gauge the new EHR system implemented at community health facilities. The results from this study showed that there was a statistically significant increase in the proportions of district health officers that experienced efficiency improvement in MCH data management, specifically for data validation and aggregation. With the introduction of the EHR system, over 50% of district health officers were able to reduce their time commitments to data management in Volta and Eastern regions in Ghana. Such benefits can motivate the district health officers to make the best use of the new system, which is scheduled for scale up to other parts of Ghana.

While the study demonstrated evidence for a reduction in time for MCH data management following the adoption of the e-Tracker, no statistically significant improvement was found in terms of data quality and use. Part of the reason for this can be explained by the high baseline level. The proportions of district health officers who had used MCH data for various tasks and had positive perceptions on data quality were already high at pre-survey, potentially implying diminishing marginal returns to scale in terms of improvement in data quality and use. In other words, the introduction of e-Tracker could produce less additional self-perceived improvement for additional extended benefits. Over 70% of the respondents said “yes” to three questions about the use of data for relevant tasks at both pre- and post-surveys. Also, over 60% of our respondents provided positive answers to the questions about data quality for both pre- and post-surveys, although consistency in accuracy across community health facilities showed mixed results.

Another explanation could be the relatively short interval between pre- and post-surveys. The pre-and post-surveys were completed over the course of 1 year, and this study was designed as a preliminary analysis for the two regions that adopted the e-Tracker during the early stages of the national roll out. Interestingly, a recent scoping review on EHR found that its effect on data quality had mixed results, with both positive and negative perceptions suggested by the previous studies.¹³ Another study on EHR from Malawi reported that the data quality was gradually improved from 2008 to 2012, implying how the improvement in data quality takes time.²⁸ Additionally, a study on the EHR implementation in Uganda provided empirical evidence on the data quality improvement by comparing the data error rates between 2007 and 2011.²⁹

As in the case of other studies for EHR systems in Ethiopia or Malawi, our study was conducted during the transition period where the paper-based health record was still used as a back-up in addition to the e-Tracker.^6,20 Therefore, further investigation is warranted for an evaluation of the e-Tracker when the duplicate effort for documentation is no longer needed, although it may require more time. In fact, a study from Ethiopia concluded that the dual documentation practice involving both paper and the EHR was a major challenge in improving the data quality.⁶

Why Data Quality, Use, and Management Practice?

This study examined the MCH data quality, use, and management practice before and after the adoption of an EHR system known as e-Tracker in Ghana, specifically from the perspectives of district level health officers. Unlike the majority of previous studies that investigated the EHR development process or clinical staff’s perspectives, this study aimed to provide empirical evidence on the potential of an EHR system for assisting public health authorities. Particularly, this study focused on data quality, use, and management because these components have been widely discussed in the previous literature on EHR evaluation, especially in the LMICs settings.

First, the data quality is an important variable for evaluating health information systems in general. Recognizing that good quality health data is essential for monitoring and decision-making, the WHO suggested a guideline for assessing certain dimensions of data quality.²⁵ These dimensions include completeness, internal, and external consistency, which we incorporated in our survey for measuring data quality.²⁵ In practice, the data quality assessment was performed as part of EHR evaluations in the context of LMICs such as in Rwanda or Kenya.^7,28

Second, the sustained and meaningful usage of a new EHR system is critical for a successful transition from paper-based record to electronic systems, as evidenced by theoretical models for continued use of the innovative technology.^30,31 Whether the end-users of an EHR system are willing to use it or not has been examined in many studies, because the acceptance or willingness can facilitate the integration of an EHR into the existing health information systems. In fact, a previous study on EHR in Malawi indicated that “low confidence to use the system among new healthcare workers” was one of the reasons that delayed the implementation for more than a year.²⁰ Another study pointed out that the use of data for visualization or analysis purposes is one of the key features of an EHR.⁵

Third, the efficiency in data management has been recognized as one of the drivers to motivate the users of an EHR system. Previous studies have demonstrated the improvement in efficiency following the adoption of an EHR system, including in Ghana. In a preliminary case study of the e-Tracker in Ghana, health workers at the community health facilities agreed that the e-Tracker system was “efficient for record keeping and data management”.⁸ Also, the implementation of an EHR helped alleviate the health workers’ burden in terms of time and administrative cost in Nigeria.³ Additionally, a scoping review on EHR suggested improved efficiency in time management among clinical staff.¹³

Contributions and Limitations of the Research

One of the key contributions of this research is that it provides evidence on the immediate effect of the EHR implementation from the perspective of district health officers. In doing so, the study attempted to demonstrate the feasibility of using a pre- and post-survey with district-level health officers for the EHR evaluation. The perceptions of key decision-makers have not been widely discussed yet in the literature on EHR in LMICs, even though they are the ones who can add value to the EHR system by using the electronic health data for action. In this sense, the survey tested in this study should be further revised and replicated for other districts in Ghana or other countries preparing for the introduction of an EHR.

As a preliminary evaluation for the EHR system in Ghana, this study has a few limitations. First, the pre- and post-survey involved self-reported responses, which may introduce the possibility of bias such as social desirability, recall bias, or misunderstanding of the questions. As suggested by previous studies on bias in self-reported responses, strategies for adjusting these can be considered in future studies.^32,33 Second, the time interval between pre- and post-surveys was relatively short, requiring further investigations in follow-up studies. Particularly, continuous effort for evaluation should be considered by involving assessment on multiple intervals or other study designs such as difference-in-differences. Third, despite the participation from almost all districts in the two regions, the sample size was not large enough for statistical modeling or advanced analysis. Therefore, future research can involve districts in other regions to address these issues when the scale up of the e-Tracker rollout in Ghana is completed.

Implications for Practice

The findings from this study can shed light on the potential value of EHR for timely decision-making and policy change by the district-level health officers in the context of LMICs. In practice however, there are still hurdles in fully realizing its potential benefits in low-resource settings. The most frequently discussed challenges in EHR implementation include poor internet connection and financial sustainability. In other words, the system level changes should also be considered in addition to the launch of EHR itself. Despite these practical challenges, this study suggests that EHR is a useful solution for district-level health officers to improve efficiency in data management.

Conclusion

This study investigated the perceptions of district health officers on the new EHR system in Ghana, using a pre- and post-survey design. Building upon previous literature that mainly assessed the effect of EHR implementation from the clinical staff’s perspectives, this research examined if the improvement that the end users experienced can also be replicated for the public health officials. On the one hand, there was a statistically significant improvement in efficiency in data management, with the reduction in time commitment. On the other hand, no significant improvement was observed in terms of data quality and use, partly because of the short interval between pre- and post-surveys and the high baseline level. Considering that district health officers are important sources for public health decision-making and resource allocations, their views on the newly adopted EHR system should be continuously monitored and evaluated. Given the ongoing progress in the e-Tracker national rollout in Ghana and the limitations of this study, this research provides a preliminary and exploratory landscape of the EHR practice at the district levels in a developing country context.