Development of Hospital MCI and Disaster Preparedness Assessment Tool for Thailand

Introduction

Disasters resulting from events due to natural and man-made hazards are occurring more frequently worldwide.¹ The World Health Organization (WHO) implemented a risk-reduction and emergency preparedness plan for the health sector and community capacity development to assess emergency preparedness for mass casualty incidents (MCIs) or disasters in each country.² MCIs and disasters are events that challenge local healthcare facilities and, potentially, entire health systems, as needs quickly exceed local capacities. This sudden imbalance occurs most typically within the first 48‒72 hours of an event before support becomes available from regional, national, and international organizations.^3,4

Hospital preparedness is a part of community resilience and is a key component to mitigate the effects of MCIs and disasters in the care of injured patients.^5,6 Recent reports indicate that many hospital emergency departments lack readiness for an MCI or a disaster, particularly those related to biochemical events.^7–9 Just as an assessment of the Emergency Medical Services (EMS) system is performed using the EMS Incident Response and Readiness Assessment (EIRRA), hospitals must assess their capacity for management of an MCI or disaster.¹⁰ Although the National Institute for Emergency Medicine of Thailand implemented its 2013‒2016 Strategic Plan for Emergency and Disaster Preparedness,¹¹ there is no recent evidence documenting the current level of hospital preparedness for MCIs and disasters in Thailand. This may be due, in part, to the lack of a standard tool that is appropriate to assess Thailand’s hospitals and the capacities of the healthcare systems.¹²

A variety of assessment tools were proposed by recognized organizations to assess hospital preparedness and readiness for either emergencies or disasters or both.^13–19 The Hospital Safety Index (HSI) proposed by the World Health Organization (WHO) is one assessment tool widely used by many countries with favorable results. However, the HSI is not applicable for use in Thailand because many items are related to hazards not present in Thailand. Furthermore, the framework of the HSI differs substantially from the health system framework of Thailand’s Ministry of Public Health (MOPH). The HSI framework includes i) hazards, ii) structural and non-structural aspects, and iii) emergency and disaster management. On the other hand, Thailand’s health system framework includes i) governance and leadership, ii) health financing, iii) health workforce, iv) information system, v) medical products and technologies, vi) service delivery, and vii) participation.

Objective

This study aimed to develop a national hospital MCI and disaster preparedness assessment tool using the evidence from national and international literature and guidelines.

Methods

Study Design

A cross-sectional study was conducted from 1 March 2015 to December 2016. Ethics approval was obtained from the Institutional Ethics Committee Board of the Faculty of Medicine at Prince of Songkla University, Thailand (REC 57-314-20-1).

Development of the Hospital Assessment Tool

The contents in the assessment tool were initially developed by performing a systematic search from electronic databases: PubMed, Medline (Ovid), Cochrane Library (Wiley), Cinahl (Ebscohost), and Embase (Elsevier), and guidelines published by the WHO and other relevant organizations in disaster management up to 31 December 2014 in both English and Thai languages. Keywords used included assessment, evaluation, protocol, hospital preparedness, guideline, and full report. These terms were combined with disaster- or mass casualty-related keywords such as cyclone, tsunami, tornado, earthquake, flood, and terrorist.

Titles, abstracts, and full texts were evaluated and assessed independently by two of the authors (PW, RR) to identify articles related to medicine, public health, hospital, and disaster management. Any disagreements between the first two authors were assessed and decided by the third author (TK). The three authors (PW, RR, TK) independently reviewed, analyzed, and extracted key elements related to hospital preparedness in all phases of a disaster cycle and further organized into a drafted hospital assessment tool. Nominal group technique was conducted to draft the final assessment tool based on Thailand’s MOPH health system framework and the six building blocks developed by WHO (ie, governance and leadership, healthcare financing, health workforce, medical products and technologies, information system, service delivery, and plus participation). Finally, the first draft of the developed assessment tool consisted of four parts: 1) general descriptive information; 2) preparedness assessment checklist; 3) suggestions; and 4) hospital actual and surge capacity (Appendix 1).

Validity Test of the Assessment Tool

The validity of the tool was measured by three recognized experts from emergency medical service organizations, provincial health offices, and attending physicians of university hospitals using an evaluation form with four domains using a 5-point scale and an open-ended suggestion. The assessment tool was then modified before presenting it at the stakeholders meeting of 46 representatives of emergency management personnel from relevant organizations that included the Department of Disaster Prevention and Mitigation, World Health Organization (Thailand office), National Institute for Emergency Medicine, and Bureau of Public Health Emergency Response, Ministry of Public Health. The meeting took the form of a public hearing to obtain inputs from the stakeholders.

Feasibility Test of the Assessment Tool

The hospital assessment tool was distributed to the hospital director, emergency manager or chief of emergency unit of 41 hospitals. A total 33 (80%) hospitals volunteered to participate in the feasibility assessment.

Evaluation of the Assessment Tool for Validity and Feasibility

The four domains of the 5-point scale evaluation form with an open-ended suggestion were (1) framework appropriateness ranging from 1 (inappropriate) to 5 (most appropriate), (2) relevance of items ranging from 1 (irrelevant) to 5 (most relevant), (3) clearness ranging from 1 (unclear) to 5 (very clear), and (4) usefulness ranging from 1 (useless) to 5 (very useful).

Data Analysis

The data were entered into an Excel spreadsheet (Microsoft Corporation). Information from the 5-point scale evaluation form was categorized into 3 scores: 1 (relevant) if a scale of either 4 or 5, 0 (uncertain) if a scale of 3, and ‒1 (not relevant) if a scale of either 1 or 2. The item-objective congruence (IOC) index was calculated by the formula IOC = ΣR/N, where ΣR = sum of scores from the experts and N = number of experts. The tool was considered valid if the results were at least 0.6. The feasibility data were analyzed and reported as median and interquartile range (IQR). The tool was categorized as feasible for median values of at least 4.

Results

Seventy-six full texts and guidelines out of 5869 titles and abstracts from a systematic search were enrolled in the study for constructive review and development of a hospital assessment tool (Figure 1).

Figure 1 Literature searched, retrieved, and the selection process.

The final drafted hospital assessment tool was composed of 127 items of a preparedness checklist: Governance/Leadership (24 items), Financing (11 items), Health Workforce (14 items), Information Systems (17 items), Medical Products and Technologies (29 items), Service Delivery (25 items), and Participation (7 items) (Appendix 1).

The IOC index values assessed by the three experts for the appropriateness, relevance of items, clearness, and usefulness were 1.0, 0.9, 0.7, and 1.0, respectively (Table 1). The response rate from the feasibility study was 80%, and the study revealed all components in the hospital assessment were feasible with a scale of at least 4. The median (IQR) values of the framework appropriateness, relevance of items, clearness, and usefulness were 4.0 (4.0‒5.0), 4.3 (4.3‒4.5), 4.0 (4.0‒4.0), and 5.0 (4.0‒5.0), respectively.

Table 1 Findings from Validity and Feasibility Tests

Qualitative information from the experts and stakeholders in the validity and feasibility tests and details of the assessment tool modification are shown in Table 2. Furthermore, valuable comments and suggestions from the validity measurement endorsed the terminology of the framework and items in the preparedness checklist as part of the hospital assessment tool. The comments and suggestions from the feasibility study were used to modify and organize the hospital assessment tool. Ultimately, the hospital assessment tool was modified to become a manual for hospital preparedness for mass casualty incidents and disasters composed of concepts, purposes and framework, evaluators, terminology of components and key elements, and the hospital assessment tool.

Table 2 Qualitative Information from the Experts and Stakeholders in Validity and Feasibility Tests and Details of Assessment Tool Modification

Discussion

Effective management of the health consequences of emergencies and disasters requires a resilient health system. The Sixty-fourth World Health Assembly has urged countries to strengthen their disaster risk-management programs by integrating them into national health systems.²⁰ Recognizing the need for strengthening health systems globally, and for all hospitals to create, revise, or update their respective disaster plans, the Pan American Health Organization and WHO developed the HSI, which is a standardized assessment tool based on an all-hazards approach and is deliberately generic in nature to allow modifications specific to the health system structure and resources of individual countries.²¹

Development of a standardized self-assessment tool based on the MOPH health system framework is needed to strengthen the hospitals in Thailand to prepare for either an MCI or disaster or both. Gaps and weaknesses need to be identified to guide hospitals to strengthen their capability and strategic planning as recommended in the systematic review of hospital disaster preparedness tools by Heidaranlu et al.²²

The current hospital assessment tool was developed by a variety of methodologies with psychometric evaluations,²² which resulted in a valid and reliable tool. The tool was more scientific, evidence-based, valid, and feasible than the tools previously developed by a panel discussion of Thai experts.²³ The systematic search provided fundamental evidence-based materials for the tool while the nominal group technique harmonized the tool’s concept and framework. The final drafted tool has suitable key elements in concordance with the country’s health system. The qualitative and quantitative assessments by the experts and stakeholders in emergency management have established the validity of the tool. The feasibility study produced a useful tool for the hospitals of Thailand. Most processes in the development and validity assessment of the hospital assessment tool were similar to the development of other tools, such as a tool to assess pediatric residents supervising inpatient rounds,²⁴ an assessment tool of indicators for quality of care in severe preeclampsia/eclampsia and postpartum hemorrhage,²⁵ and a forensic recording form for firearm injuries.²⁶

The tool is concise and composed of the key elements in MCI and disaster management (eg, structure, functions, risk assessment, emergency and disaster management, actual and surge capacity, and participation).^{13–19,27,28} The tool is compatible with the HSI developed by WHO with arrangement of the items based on the MOPH health system framework and the six building blocks developed by WHO. The tool can be used for self-assessment of hospital preparedness and can help hospitals recognize their readiness, identify gaps for improvement as required, and allow for inputs for strategic planning. The six building blocks developed by WHO for a health system framework has been used in many countries and has proven to improve mass casualty and disaster management.^29–32

Analyzing hospital preparedness and readiness for all-hazards mass casualty incidents and disasters by conducting a national survey would provide valuable information both to hospitals for better preparation and the MOPH of Thailand for policy guidance. The tool presented in the study can be applied in any nation with a similar health system framework to strengthen hospital preparedness and promote community resilience. Furthermore, for sustainable development and improvement, the tool needs to be incorporated into the hospital safety concepts outlined by Thailand’s MOPH and/or other organizations, especially the Hospital Accreditation Organization. A self-assessment scoring tool will be developed and a pilot feasibility study will be conducted. The result will compare the level of hospital preparedness for MCI and disaster among hospitals and provide guidance for a development of more comprehensive hospital assessment tool for MCI and disaster.

Limitations

This study has two limitations. First, publication bias was possible from the use of electronic databases that may not have contained unpublished but potentially useful tools or guidelines. Second, the response rate of only 80% from the feasibility test was used to finalize the assessment tool.

Conclusion

A hospital MCI and disaster preparedness assessment tool for Thailand was developed. The tool was valid and feasible to evaluate the current readiness of hospitals in Thailand. Use of the tool will provide essential information to strengthen the country’s health system and serve as a model in other low- to middle-income countries.