Health care professionals’ knowledge and awareness of the ICD-10 coding system for assigning the cause of perinatal deaths in Jordanian hospitals

Introduction

Vital registration data systems are necessary for countries to identify and quantify health-related issues. These systems provide accurate, complete, and timely vital statistics that can be used to measure progress toward quality improvement and meeting public health goals.^1,2 Perinatal and neonatal (PNN) mortality and its risk factors provided by such systems serve as particularly important socioeconomic and health indicators, thereby influencing policy development to prevent stillbirths and improve research on their interventions and high-risk factors.^3–5 Hence, accurate and timely registration of births and deaths is an essential feature of high-quality vital registration systems.^1,5

Despite the usefulness of accurate and well-structured registration systems for reporting PNN deaths and their causes,^5,6 there is a lack of such systems in low- and middle-income countries, including Jordan. Available data on PNN mortality in Jordan come from periodic surveys or from national vital registration statistics. These data do not permit robust analysis of causes of death and associated risk factors and do not usually include quality indicators to help in planning and monitoring PNN services. Registration of births and deaths is based merely on written reports by family members who are largely unaware of the need for early reporting; therefore, many neonate deaths are not officially reported. A recent study conducted in Jordan⁷ identified many problems in the registration of births and neonatal deaths, including a high rate of under-registration of neonatal deaths and no registration of stillbirths. Another study showed that the death notification form including causes of death is not usually completed by physicians, that physicians are not well trained on how to assign causes of death, and that physicians are often not aware of distinctions between direct, underlying, and contributing causes of death.⁸ It is clear that Jordan lacks sufficient credible evidence for effective health planning. Thus, an electronic surveillance system that automatically transfers the data of PNN deaths and their causes to the Ministry of Health (MoH), allowing them to measure progress in quality improvement, would be invaluable for policy makers.

Identifying the underlying cause of death is multifaceted, as detailed in the International Classification of Diseases, 10th version (ICD-10).⁹

The ICD-10 is the standard international classification that is used to report mortality data in order to provide reliable epidemiological information.⁹ ICD-perinatal mortality (ICD-PM) is a new program-based system, which is derived from the ICD-10 and which adheres to the rules and instructions of mortality coding of the ICD-10. The main aim of this system is to direct interventions toward decreasing perinatal deaths.¹⁰

Identifying and categorizing the primary cause of death can be challenging due to the ICD-10 rules.⁹ The quality of the register that reports the cause of death is largely influenced by the quality of the information documented in the death certificate.¹¹ Several factors can affect the quality of documentation, including the reliability and accuracy of diagnostic procedures and the precision taken in documenting the death.

Unfortunately, to date, there is a lack of consistency and accuracy in the documentation of the cause of death, especially in the absence of international standards.¹² Particularly in low- and middle-income countries, documentation and reporting of perinatal deaths is inadequate,¹³ and research conducted in such countries revealed a failure in adequate examination of the causes of perinatal deaths, especially in regions of high perinatal mortality rates.¹⁴ Similarly, identifying the underlying causes of perinatal deaths is a difficult task when there are inadequate resources and a lack of clarity at the health care settings in which the deaths occur.¹⁵ This is worrisome, especially given that the majority of stillbirths occur in the intrapartum stage and about three quarters of neonatal deaths can be avoided.^16,17

Another critical factor in the success and effectiveness of any death registry is the level of health care professionals’ (HCPs’) awareness and attitudes toward it. Research has shown that there are negative attitudes toward the implementation of coding systems by health care providers and coders, and this, in turn, could make the implementation process harder. The complexity of coding systems and the lack of skilled coders might affect HCPs’ use and compliance to such systems.^18,19 It has been found that older physicians are more reluctant to use electronic health record (EHR) systems, while younger doctors and doctors who work in larger hospitals are more open to the implementation of such systems.^20,21

Based on the Higher Health Council’s recent report “National Strategy for Health Sector in Jordan 2015–2019” the most important challenges facing the health system in Jordan include lack of credible evidence for effective planning and decision-making in the health system, lack of a national strategy for health information and research data, weakness in the computerization of health specialized data, and weakness in the application of electronic Modern Health Systems Applications.²² The availability of PNN mortality electronic surveillance systems in Jordan is important for accountability, decision-making, planning, and development of effective policies and strategies for PNN care. In order to build an effective PNN mortality electronic surveillance system that provides necessary data, it is important to first understand the health professionals’ awareness, knowledge, and use of the ICD-10 coding system and the perceived barriers to the implementation of this system. Therefore, the current study aimed to assess health professionals’ awareness, knowledge, and use of ICD-10, assess their perceived barriers to the use of ICD-10, and assess their perceptions of electronic perinatal death surveillance systems in general.

Participants and methods

The current study used both quantitative and qualitative approaches to enhance the trustworthiness and credibility of the findings.

Quantitative part

A descriptive cross-sectional study was conducted in four major hospitals from different geographical areas in Jordan. These four hospitals provide health services to a large segment of the population. Three hospitals are the largest maternity and pediatric hospitals in the north and north east while the fourth hospital is the largest in the south of Jordan. Birth and death registrations are usually handled by pediatric physicians and registered nurses. Thus, all physicians and registered nurses working in pediatric departments in the four hospitals were invited to participate in the study, and those who agreed (response rate >85% in all hospitals) were interviewed by the trained nurses using face-to-face structured interviews. The quantitative part of the current study included a total of 84 pediatricians/residents and 218 nurses from the four selected hospitals.

The study questionnaire collected information from HCPs through three main sections that contained closed-ended questions; one section was about HCPs’ sociodemographic data, one about familiarity, awareness, knowledge, use, and barriers to the use of ICD-10, and one included three case studies. The case studies aimed at assessing HCPs’ skills and abilities in determining the direct and indirect/contributing causes of deaths that are related to the mother and/or fetus or the neonate, according to WHO guidelines.²³ These case studies were adopted and used previously to certify the cause of death.^24,25 All HCPs were asked to read the case studies carefully and then write down the answers on the space provided below each case study without using any aids or accessing any information (related websites, books, or booklets with information on the ICD-10). Data were analyzed using SPSS version 20 (IBM Corporation, Armonk, NY, USA). Data were described using percentages for categorical variables and mean and SDs for continuous variables.

Qualitative part

A qualitative approach using focus group discussions was also utilized to explore deeply the HCPs’ experiences, perceptions, and knowledge of coding systems and of the electronic perinatal death surveillance system. A total of 16 focus group discussions were conducted between March 2018 and May 2018 in the four major representative hospitals in Jordan. Four focus group discussions were conducted in each hospital with HCPs who were pediatricians, obstetrics and gynecologists, senior residents, registered nurses, and midwives. An average of 5 HCPs were interviewed in each focus group, with a total of 80 HCPs participating in the 16 focus group discussions. We made sure that each focus group included at least one HCP from a different specialty.

Data collection

These HCPs were asked to discuss their knowledge and awareness of the ICD-10 coding system and their perceived benefits and challenges of the current electronic medical record in the hospital they worked at. Health care providers’ perceptions were derived from the group discussions using a group-interview guide, and the discussions were developed using open-ended questions to facilitate a consistent data collection approach during the in-depth group interviews. Each group discussion started with a general question about the HCPs’ awareness and perceptions of the current reporting and registration process of neonatal deaths and stillbirths, and then proceeded to more detailed questions based on the answers and experiences shared by the participants in order to elicit discussion. The interviews were conducted in Arabic, the local language, with some use of English medical terms by the HCPs during the discussion.

Ethical considerations

Ethical approval was sought and obtained from the Institutional Research Committee of Jordan University of Science and Technology and from the MoH for the qualitative and quantitative parts of the current study. After full clarification of the study aims and ethical considerations, all participants signed the informed consent form.

Data synthesis and analyses

All recorded interviews were transcribed in full verbatim and then checked for accuracy by the project team who attended the group discussions. The whole content thematic analysis process was done in its original Arabic language to conserve the credibility of the findings. Analysis of the transcribed data was undertaken manually through a coding process and the generation of categories and themes. The content thematic analysis approach consisted of a full reading of the whole transcripts by two independent researchers from the project team, followed by highlighting and coding important sentences into different categories several times until themes and subthemes were identified and agreed upon by the two researchers.

Translation of themes as well as quotes into English was undertaken after the generation of themes and subthemes. Preliminary analysis was conducted after each interview in order to obtain a general impression of the data, which allowed for early identification of gaps in the interviews that sometime required further clarifications from the HCPs.

Results

Quantitative part

Participants’ characteristics

The quantitative part of the current study included a total of 84 pediatricians/residents and 218 nurses working in the four selected hospitals. More than half (53.0%) of the physicians were females and almost all the nurses were females. Their ages ranged from 17 to 62 years with a mean (SD) of 31.1 (6.5) years, and their years of experience ranged from 1 to 35 years with a mean (SD) of 7.6 (6.5) years and a median of 5.0 years.

Knowledge and use of ICD-10

Table 1 shows that only one-third (39.3%) of the physicians and one-quarter (24.8%) of the nurses were able to mention what ICD-10 stands for. Only 34.6% of the physicians and 14.7% of the nurses were familiar or somewhat familiar with the ICD-10. The findings also show that only about 6.0% of the physicians had used the ICD-10 very well before and 16.7% had used it with difficulty, and a much smaller number of the nurses had used it very well (0.9%) or with difficulty (3.7%). Furthermore, only 11.9% of the physicians and 5.5% of the nurses had received training on ICD-10. Overall, 70.2% of the physicians and 83.5% of the nurses did not know whether or not their departments used ICD-10 to record causes of perinatal deaths. The knowledge and the self-reported use of ICD-10 did not differ significantly according to age when health professionals’ age was categorized into <30 vs ≥30 years.

Table 1 HCPs’ awareness and knowledge of the ICD-10 Abbreviations: HCP, health care professional; ICD-10, International Classification of Diseases, 10th version.

The primary obstacles/barriers to the use of ICD-10 by maternities

Table 2 shows that about one-quarter (23.2%) of the physicians and nurses cited lack of knowledge about ICD-10 as being the main barrier to the use of the ICD-10. Lack of an electronic system to support ICD-10 use was cited as being the second main barrier to the use of ICD-10 by both physicians and nurses. Lack of time, staff, and support by the administration were all reported as barriers by 8.3%, 7.3%, and 4.3% of the participants, respectively. The results revealed that physicians were more likely than nurses to report these barriers, especially the barrier of lack of staff interest.

Table 2 The main obstacles/barriers for the maternities to use ICD-10 Abbreviation: ICD-10, International Classification of Diseases, 10th version.

Neonatal death notification and perceptions of electronic perinatal death registration

Our results show that only 30.9% of the physicians and 14.0% of the nurses reported that they had received training on filling the death certificate and assigning the causes of death. About 41.7% of the physicians and 26.1% of the nurses did not know who was responsible for the registration and notification of deaths in their hospitals. When they were asked about the importance of having a new system for the electronic registration of births and neonatal deaths, 86.9% of the physicians and 85.2% of the nurses saw that it was important to have such a system. Less than half of the physicians (45.2%) and 35.0% of the nurses stated that they would definitely use the system to register births and deaths, and 39.3% of the physicians and nurses reported that they would probably use it. Furthermore, 56% of the physicians and 65.3% of the nurses stated that the workplace policies ensured security and confidentiality of the data.

Assigning causes of neonatal deaths according to ICD-10

Table 3 shows that the percentages of physicians who correctly identified the main diseases or conditions that lead to death in newborns in the three cases (scenarios) were 51.2%, 29.8%, and 67.9%, respectively. For nurses, the percentages were 60.1%, 30.3%, and 68.8%, respectively. A smaller number from both groups were able to identify other possible diseases or conditions that contribute to death in newborns. Furthermore, the table shows, a much smaller number of participants were able to identify the main and the other maternal diseases or conditions affecting newborns. It appeared that the nurses were just as good as, if not better than, the physicians at correctly identifying causes of deaths.

Table 3 The percentage of health professionals who correctly identified the causes of neonatal deaths

Qualitative part

Knowledge and awareness of the ICD-10 coding system

In congruence with the quantitative results, the majority of HCPs across all the hospitals were not aware of or familiar with the ICD-10 coding system. Some physicians acknowledged that they never heard about ICD-10. “This is the first time I’ve heard of it […]. I never heard about ICD-10” (A physician in GD3).

However, nurses who work in Quality Assurance departments and some senior residents were familiar with the system. A quality assurance nurse manager stated, “I’m familiar with it as I work at the Quality Assurance department and we use the ICD-10 for accreditation purposes […]. However, we (referring to the nurses) are not required to use it […]”. Most specialist physicians, especially old ones, tended not to use the system and reported that it is usually the senior resident who fills out the electronic files/templates.

“I do not use it. I guess that the residents are the ones who document the cause of death most of the times”, reported a senior pediatrician.

There were several reasons behind HCPs not using the coding system, as mentioned by several participants. These reasons included HCPs’ unawareness of how the coding system works and its importance, the large amount of clinical work involved, and the time needed to assign the cause of death, which can often be a burden on the doctor. Additionally, several doctors complained that the coding system is not comprehensive. One senior resident said, “it doesn’t always have the exact cause/diagnosis that we identify […] we’re forced to write the diagnosis manually or choose the closest diagnosis but not the exact one”.

In the University Hospital, doctors said that they had heard about the ICD-10 but that they usually wrote the diagnosis “encounter” manually and then the Medical Records department used the coding system to match the diagnosis with the appropriate codes. However, most of the interviewed physicians at the University Hospital were concerned that the coders at the Medical Records department were not trained enough or familiar with the ICD-10 coding system, thus meaning that they may sometimes choose the wrong codes for diagnosis. Another related concern raised by the University Hospital physicians was that the coders rely on what the doctor writes in his/her notes about the diagnosis. If the doctor writes a wrong diagnosis, then the coders will, therefore, choose a wrong encounter. One pediatrician at the University Hospital mentioned, “for example, if I diagnose the baby wrong and write it in my discharge notes, the guys at the Medical Records department will insert a wrong code […]”.

Another senior resident at the same hospital added, “we try our best to use terminologies that match those found in the ICD-10 […] but not all doctors adhere to these terminologies…this makes it hard for the coders in the Medical Records departments to find the appropriate code for the diagnosis”. In some cases, where the cause of death is “unspecified” in the death certificate completed by the doctor, the coders then choose any encounter randomly, even if it is wrong. In this sense, a senior pediatrician stated, “we have tried to train residents on this coding system at least, to decrease the cases of wrong or unspecified diagnoses”.

This issue was not shared by the physicians working at the governmental MoH hospitals, as the doctors themselves who write the diagnosis may also use the ICD-10 codes, although the majority of the doctors do not use it.

“Honestly, we do not use the ICD-10 despite it being there on the system…because we do not memorize the codes of different diagnosis/encounters”, stated a senior resident at a MoH hospital. However, another resident said, “we started using the coding system ever since the electronic medical file was implemented at our hospital, although we do not always find the diagnosis that we think apply in certain cases”.

Perceived problems and challenges with the current electronic patient medical systems

Lack of adequate training and familiarity with the system, especially for junior staff, as well as high turnover rates of staff and transfer of doctors between hospitals that do not have electronic medical record systems were the main challenges mentioned by HCPs working in the MoH hospitals. One critical problem, mentioned by HCPs working at the three MoH hospitals, was the defect in “electronic” communication between hospitals, especially in cases of patient transfer from one hospital to another and despite the fact that the hospitals are connected to the same electronic medical system. On top of that, HCPs complained that the current electronic medical record does not generate statistics about birth rates, death rates, etc. One specialist physician said, “it doesn’t produce useful numbers that I can look at and benefit from […] which makes it not efficient and useless”. Another senior resident added, “it’s just not smart enough […]. I mean it’s not an intelligent system that saves information […] and it’s boring”.

With regards to the infrastructure, the majority of HCPs who use the current patient medical record at the hospitals they work at complained that they have old computers and printers, which are slow and have poor access to the internet. A doctor at one hospital admitted, “I’d rather see 60 patients in my clinic and provide care for them […] than write anything via the electronic medical file as it’s such a waste of time and it’s old”.

With regards to the registration process of any neonate on the current electronic medical record at MoH hospitals, a number is assigned to the newborn in his/her electronic medical file at the time of delivery. However, if the parents register their neonate officially in the Civil Status department, where the baby is given a unique national ID, then the original electronic file for the baby that was generated at the time of delivery is deleted automatically from the current electronic medical system unless integration of the two files occurs, which is very rare. However, in the University Hospital, staff use “another electronic software”, meaning that any newborn who is admitted to the NICU at the University Hospital is given a unique serial number and a medical file via the electronic system that can be chased even after the newborn has been registered officially in the Civil Status department and has obtained a national ID. However, newborns who are transferred from a MoH affiliated hospital into the University Hospital cannot have their electronic medical record available/accessible to doctors and nurses at the University Hospital, and this makes it hard to provide proper care due to the lack of “electronic” medical history of the neonate.

Discussion

Overall, there was congruency between both the quantitative results and the qualitative findings which arose from the current study. The majority of HCPs in the four selected hospitals were not familiar with the ICD-10 coding system and hence reported minimal use of it. Surprisingly, the majority of HCPs were not aware of whether or not their department used the ICD-10 to record perinatal mortality. HCPs identified lack of knowledge, time, staff and management support, and an effective and comprehensive electronic system that can accurately allow physicians to select the exact cause of death as main barriers to the use of the ICD-10.

The findings from previous literature are consistent with our findings. On one hand, several worries were reported in the literature by doctors on the use of ICD-10, such as the readiness and comprehensiveness level of the software itself, the time and precision needed for documentation, doctors’ capacity to learn a new, unfamiliar skill that is relevant to computer software, and lack of training.²⁶ On the other hand, doctors reported some benefits of implementing the ICD-10-CM, including effective data statistics.²⁶

Moreover, also in congruence with our findings, previous literature identified negative attitudes of health care providers and coding professionals toward the implementation of ICD-10-CM/PCS.¹⁸ These negative attitudes were due to a lack of adequate coding professionals and difficulties in dealing with a complex coding system, which can be frustrating for some health care providers.¹⁹ Additionally, junior doctors were more open to the use of EHR systems, whereas older doctors were more reluctant to using such systems.²⁰ Regardless, the commitment levels shown by health care providers to the use of the new coding system can predict the success of the implementation process.²⁷ Moreover, few nurses and physicians in the current study reported having received training on filling out the death certificate and assigning the causes of death electronically in their hospitals. The vast majority of HCPs encouraged the idea of having an electronic registration system to record all perinatal deaths; however, less than half showed intention on using the system to record and register these deaths.

Surprisingly, one-third of HCPs were not able to identify the correct main cause of death in case study 1, which is still a higher percentage among the other two case studies, yet a smaller number of them were able to identify the contributing causes of death and the maternal conditions linked to the death of the newborn. Studies reported that cause of death statistics can permit comparisons within and between countries and can support the development and evaluation of appropriate health interventions.¹¹ However, identifying the underlying cause of perinatal death is complex due to lack of resources and lack of precision at hospitals.¹⁵ An improvement in this is critical, especially given that the majority of perinatal deaths are usually preventable.^16,17

The majority of the HCPs in the current study reported lack of training on how to fill out a death certificate, and they were not sure who was responsible for completing the death certificate electronically. The information written on the death certificate is detrimental for accurate registration on the electronic system/software.²⁸ However, similar to our findings, some HCPs still report more than one underlying cause of death, and this can be a challenge to the prevention of similar future deaths.²⁸ Several factors may lead to the incorrect registration of the cause of death, such as inappropriate documentation of the death certificate by the doctor, personal interpretation of medical terms, complexity of the ICD-10 coding system, and diverse understanding of coding rules. Hence, one of the vital factors of improving the accuracy and consistency of the cause of death statistics is the proper use of the standardized coding system (ie, ICD-10).²⁸ Furthermore, implementing a new coding system in hospitals requires periodic and comprehensive training on computer and the coding systems as well as effective management and leadership skills.²⁹ Research has shown that about half of HCPs stated that training was the most important element of the ICD-10-CM/PCS conversion.³⁰

Sweden provides an example of a two-stage process of reporting the death and issuing the death certificate.¹¹ First, the doctor who witnesses and confirms a death needs to report the death to the Swedish Tax Agency straight away. This step is very important to permitting authorization of burial, even if the cause of death is not written in the “notification of death” report. Second, another written report, such as the medical death certificate, should be submitted to the National Board of Health and Welfare within 3 weeks of the death. Information from this death certificate is included in the documentation of the cause of death register.¹¹ It is worth noting that, in Sweden, the death certificate is usually written by the patient’s doctor or the doctor who last saw the patient before death.

Similar to the Swedish register, a successful and strong death register for the cause of death needs to be comprehensive, used over a long period of time to allow retrieval of patient health history, and needs to use a well-developed software to help in the coding process of classifying the underlying cause of death.¹¹ This will result in sophisticated data that permit comparisons with countries using similar software.

Accurate classification of the causes of perinatal deaths across all settings, using a well-known, internationally recognized system is the fundamental step in selecting and using relevant software and programs.²³ Both perinatal and maternal conditions are highly related; hence, aiming at decreasing perinatal mortality could also help in reducing maternal mortality and vice versa.²³ This can be best achieved through developing a system that focuses on the underlying causes and the perinatal and maternal conditions contributing to death simultaneously.

Although some of the HCPs in our study said that they might not use the electronic death registry once developed, having a standardized program that can accurately identify and document the causes of death, both direct and indirect, has a great potential to facilitate interpreting the perinatal and maternal data linked to deaths.²³ The transition to a new coding system will permit health care providers to categorize diseases, improve documentation of diagnosis and any related complications, and monitor and evaluate outcomes related to health care more successfully, especially in rural and disadvantaged regions.^8,18

Accordingly, regular training of HCPs on how to use the ICD-10 is vital during the application of any new health technology system.²³ Therefore, stakeholders and managers of hospitals need to hire and continuously train new coding professionals to implement the system in a quick and sustainable way.¹⁹ For doctors, nurses, and midwives, the transition to ICD-10 requires a change in how patients’ medical history and diagnosis are documented.²⁷ HCPs need to find out how the new coding system will influence their capability to correctly recognize diagnosis and process codes and to be detailed and specific. Moreover, training workshops for the ICD-10 coding system need to pay particular attention to the importance of obtaining and reporting complete and accurate medical information,³¹ and must aim at increasing HCPs’ awareness levels of the characteristics and benefits of the ICD-10 coding system.^30,31 Calculating the rate of stillbirths and neonatal deaths and collecting the necessary information about the direct and indirect causes and contributing factors of death enable HCPs and stakeholders to lead the death review and audit process to the prevention of similar future deaths.^23,32

Finally, the use of a survey could limit the trustworthiness of the study, however, to overcome this limitation, we used a mixed method design in which we conducted 16 different focus group discussions to ensure credibility of the quantitative part. Public hospitals in Jordan are relatively new in using the EHRs including the ICD-10, which could have limited the HCPs understanding of the topics being discussed.

Conclusion

The current findings emphasize the importance of having an effective and comprehensive electronic system to accurately report and register all perinatal deaths. The system also needs to take into consideration the direct and indirect causes of death and the contributing maternal conditions at the time of perinatal death. Implementing a new coding system in a health care organization requires adequate awareness sessions and periodic training of HCPs on computers and the coding system, as well as effective management and supportive leadership.

Acknowledgments

This work was supported and funded by the International Development Research Centre/Canada and The United Nations International Children’s Emergency Fund (UNICEF).

Disclosure

The authors report no conflicts of interest in this work.

References

1.		American College of obstetricians and Gynecologists. The importance of vital records and statistics for the Obstetrician-Gynecologist: ACOG Committee opinion no. 748. Obstet Gynecol. 2018;132:e78–e81.
2.		Howland RE, Li W, Madsen AM, et al. Evaluating the use of an electronic death registration system for mortality surveillance during and after Hurricane sandy: New York City, 2012. Am J Public Health. 2015;105(11):e55–e62.
3.		Sugai MK, Gilmour S, Ota E, Shibuya K. Trends in perinatal mortality and its risk factors in Japan: analysis of vital registration data, 1979–2010. Sci Rep. 2017;7(1):46681.
4.		Paneth N, Thompson T. 1 – Neonatal and Perinatal Epidemiology. In: Gleason CA, Juul SE, editors. Avery’s Diseases of the Newborn (Tenth Edition). Elsevier; Philadelphia, PA: 2018:1–10.e2. Available from: https://books.google.jo/books?hl=ar&lr=&id=zRNBDwAAQBAJ&oi=fnd&pg=PP1&dq=Neonatal+and+Perinatal+Epidemiology.+In:+Gleason+CA,+Juul+SE,+editors.+Avery%27s+Diseases+of+the+Newborn&ots=5c_Qbju-qB&sig=zedlIwzJONUfeG8HxAdAtx8uR6g&redir_esc=y#v=onepage&q&f=false. Accessed Agust 30, 2018.
5.		Anwar J, Torvaldsen S, Sheikh M, Taylor R. Under-estimation of maternal and perinatal mortality revealed by an enhanced surveillance system: enumerating all births and deaths in Pakistan. BMC Public Health. 2018;18(1):428.
6.		Bailey PE, Andualem W, Brun M, et al. Institutional maternal and perinatal deaths: a review of 40 low and middle income countries. BMC Pregnancy Childbirth. 2017;17(1):295.
7.		Batieha AM, Khader YS, Berdzuli N, et al. Level, causes and risk factors of neonatal mortality, in Jordan: results of a national prospective study. Matern Child Health J. 2016;20(5):1061–1071.
8.		Khader Y, Al-Sheyab N, Alyahya M, Batieha A. Registration, documentation, and auditing of stillbirths and neonatal deaths in Jordan from healthcare professionals’ perspectives: reality, challenges and suggestions. J Matern Fetal Neonatal Med. 2018;11(6):1–11.
9.		World Health Organization. International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Volume 1. Geneva: World Health Organization; 2016. Available from: http://www.who.int/iris/handle/10665/246208. Accessed August 13, 2018.
10.		World Health Organization. The WHO Application of ICD-10 to Deaths During the Perinatal Period: ICD-PM. Geneva: World Health Organization; 2016. Available from: http://apps.who.int/iris/bitstream/handle/10665/249515/9789241549752-eng.pdf?sequence=1. Accessed March 16, 2018
11.		Brooke HL, Talbäck M, Hörnblad J, et al. The Swedish cause of death register. Eur J Epidemiol. 2017;32(9):765–773.
12.		Johansson LA, Westerling R, Rosenberg HM. Methodology of studies evaluating death certificate accuracy were flawed. J Clin Epidemiol. 2006;59(2):125–131.
13.		World Health Organization. Every Newborn: An Action Plan to End Preventable Deaths. Geneva: World Health Organization; 2014. Available from: http://apps.who.int/iris/bitstream/handle/10665/127938/9789241507448_eng.pdf?sequence=1. Accessed August 17, 2018.
14.		Aminu M, Unkels R, Mdegela M, Utz B, Adaji S, van den Broek N. Causes of and factors associated with Stillbirth in low- and middle-income countries: a systematic literature review. BJOG. 2014;121(Suppl 4):141–153.
15.		Allanson ER, Tunçalp Ӧ, Gardosi J, et al. Optimising the International Classification of diseases to identify the maternal condition in the case of perinatal death. BJOG. 2016;123(12):2037–2046.
16.		Bhutta ZA, Das JK, Bahl R, et al. Can available interventions end preventable deaths in mothers, newborn babies, and stillbirths, and at what cost? Lancet. 2014;384(9940):347–370.
17.		de Bernis L, Kinney MV, Stones W, et al. Stillbirths: ending preventable deaths by 2030. Lancet. 2016;387(10019):703–716.
18.		Meyer H. Coding complexity: US health care gets ready for the coming of ICD-10. Health Aff (Milwood). 2011;30(5):968–974.
19.		Schwieters J. Strategies for dealing with the National coding shortage. Healthc Financ Manage. 2010;64(4):36–38.
20.		DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care – a national survey of physicians. N Engl J Med. 2008;359(1):50–60.
21.		Bae J, Encinosa WE. National estimates of the impact of electronic health records on the workload of primary care physicians. BMC Health Serv Res. 2016;16(1):172.
22.		World Health Organization. The National strategy for health sector in Jordan 2015–2019. Geneva: World Health organization; 2014. Available from: https://jordankmportal.com/resources/jordan-national-health-sector-strategy-2015-2019. Accessed March 13, 2018
23.		World Health Organization. Making Every Baby Count: Audit and Review of Stillbirths and Neonatal Deaths. Geneva: World Health Organization; 2016. Available from: http://apps.who.int/iris/bitstream/handle/10665/249523/?sequence=1. Accessed March 13, 2018
24.		World Health Organization. International Statistical Classification of Diseases and Related Health Problems: Tenth Revision, Volume 2, Instruction Manual. Geneva: World Health Organization; 2011. Available from: https://www.who.int/classifications/icd/ICD10Volume2_en_2010.pdf. Accessed February 20, 2018.
25.		Rampatige R, Riley I, Gamage S, Paoin W, Upham S. Handbook for Doctors on Cause-of-Death Certification. HIS Knowledge Hub Capacity Building Tools Series Brisbane: University of Queensland School of Population Health; 2012.
26.		Watzlaf V, Alkarwi Z, Meyers S, Sheridan P. Physicians’ outlook on ICD-10-CM/PCS and its effect on their practice. Perspect Health Inf Manag. 2015;12:1b.
27.		Sanders TB, Bowens FM, Pierce W, Stasher-Booker B, Thompson EQ, Jones WA. The road to ICD-10-CM/PCS implementation: forecasting the transition for providers, payers, and other healthcare organizations. Perspect Health Inf Manag. 2012;9:1f.
28.		Grippo F, Grande E, Simeoni S, et al. Reliability of causes-of-death statistics: the Italian experience from the ICD-10 training course. Rivista Di Statistica Ufficiale. 2015;17:103–119.
29.		Rodrigues R.J. Compelling issues for adoption of e-health. The Commonwealth Ministers Preference book [Internet]; 2008. Available from: http://ehealthstrategies.comnehealthstrategies.comnxxx.ehealthstrategies.com/files/Commonwealth_MOH_Apr08.pdf. Accessed March 20, 2018
30.		Jackson VE, Muckerman A. Navigating regulatory change: preliminary lessons learned during the healthcare provider transition to ICD-10-CM/PCS. Perspectives in Health Information Management/AHIMA, American Health Information Management Association. 2012;9. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3329201/pdf/phim0009-0001d.pdf. Accessed March 20, 2018
31.		Handy L. Provider Education and Training for ICD-10 Implementation. New Perspectives on Healthcare Risk Management, Control & Governance; 2014;33:37–40. Available from: http://eds.a.ebscohost.com/abstract?site=eds&scope=site&jrnl=21551138&AN=97578184&h=fWA6Z3HQilHx10O%2bEt2Gs7HjXgBg1XPt29dHIZa5AnG0YKGmXgLq6eBQ2by4lpt3XHlzdEvOPpf6ApejU3MaLw%3d%3d&crl=c&resultLocal=ErrCrlNoResults&resultNs=Ehost&crlhashurl=login.aspx%3fdirect%3dtrue%26profile%3dehost%26scope%3dsite%26authtype%3dcrawler%26jrnl%3d21551138%26AN%3d97578184. Accessed April 13, 2018.
32.		Aminu M, Bar-Zeev S, van den Broek N. Cause of and factors associated with stillbirth: a systematic review of classification systems. Acta Obstet Gynecol Scand. 2017;96(5):519–528.