Translation, Cross-Cultural Adaptation, and Measurement Properties of the Portuguese Version of the Global Trigger Tool for Adverse Events

Introduction

In recent years, patient safety (PS) and improved health care quality have been at the forefront of health policies. Although there was notable progress in understanding the frequency, causes, and consequences of adverse events (AE), its magnitude is still unknown,¹ thus constituting an important public health problem.²

One of the axes of PS is to ensure that the care provided to patients does not cause harms, injury, or complications³ to the natural disease progression and to guarantee the necessary care, such as therapeutic or palliative management for a well-timed correct diagnosis.⁴ However, as health technologies become more sophisticated and health units more complex, the risk of AE increases for patients.⁵

The World Health Organization defines AE as a physical, social or psychological incident that damages the structure or function of the patient’s body, or any resulting harmful effects, including illness, injury, suffering, disability, or death.³

In recent years, studies on the nature and impact of AEs have reported extensive prevalence differences. For example, some researchers state that about 10–12.5% of the hospitalized patients are victims of AE^6–8 resulting from healthcare provision,¹⁰ and others identified AE rates of 20.5–44.6%.^10–13 The most frequent types of AE reported were surgical injuries, medication errors, healthcare-related infections, and allergic reactions.^2,8 These important differences in AE prevalence are probably due to the different methods and designs used.

For this reason, the continuous general performance improvement process of health units should be a permanent strategic objective focused on the development of methodologies to improve the quality of care provided, including the analysis of AE causes and the study of its consequences.^9–10

Despite some improvements in PS, scientific evidence shows that patient harms persists,¹¹ being limited by the lack of advances in AE measurement and detection accuracy, since today’s health systems are limited to measuring incidents and AE using traditional methods.¹² In most situations, AE measurement is based on voluntary notification systems, audits, analysis of processes coded in Homogeneous Diagnostic Groups (HDG), monitoring of clinical indicators, complaints, and patient’s reports.

These methods can be fundamental instruments in PS if these incidents and AE are communicated and analyzed.¹³ However, the literature shows that most incidents are not reported by health professionals,¹⁴ with underreporting¹⁵ since only 10–20% of the AEs are reported.^16–18 Likewise, the codification of clinical processes is affected by the quality of the records made by health professionals,^17,18 and thus the analysis of AEs becomes insufficient.

Therefore, health institutions must know the reality to provide high-quality care, learning from their mistakes and implementing valid, sensitive, cost-effective, and easy-to-use instruments and protocols^2,13,19 that can provide useful and reliable real-time data.^12,14,20,21

The Institute for Healthcare Improvement (IHI) developed the Global Trigger Tool (GTT-IHI),²² which showed international promise in identifying and quantifying patient harm severity, consequently decreasing the occurrence of AE.^21,23,24

In Portugal, a hospital institution used this tool for risk management, and the Portuguese Association for Hospital Development (Associação Portuguesa de Desenvolvimento Hospitalar – APDH) promotes it in training sessions aimed at professionals from various health units in the country. However, no studies on its validity and reliability were found.

Thus, the introduction of the present tool validates and adapts to the identification of AEs in the Portuguese context, namely in medical-surgical services, would be another important benefit for the knowledge of the true magnitude and impact of AEs. And so, by recognizing them, it became easier to implement measures and programs for continuous improvement in risk management.

Considering the great relevance of introducing new valid tools to evaluate and analyze AEs, the objective of this study was to adapt, validate and analyze the measurement properties of GTT-IHI for the Portuguese context to use in the medical-surgical departments of Portuguese hospitals.

Methods

A retrospective cross-sectional study based on a random sample of medical records of patients admitted to a medical service at the Algarve University Hospital Center (Centro Hospitalar Universitário Algarve – CHUA). The study was approved by the administration and hospital research office after the positive opinion of the CHUA Health Ethics Committee and the process of cultural translation, adaptation, and validation of the IHI-GTT was authorized by the main author.²² Patient consent was not sought from the Ethics Committee for review of clinical medical records. All procedures were mentioned and the principle of data anonymity and confidentiality was guaranteed by coding medical records. The study was conducted in compliance with the Declaration of Helsinki.

The Instrument

The GTT methodology is based on a retrospective review of medical records using triggers to identify patient harms resulting from possible AE. The GTT contains six modules that integrate several triggers: general care (15), surgery (12), intensive care (5), medication (13), perinatal (4), and emergency (3).²²

The tool uses five categories of AE damage to the patient: temporary harm requiring intervention (E), temporary harm requiring prolonged hospitalization (F), permanent harm (G), interventions required to sustain life (H), and death (I).

According to the IHI, these processes should be reviewed by a group of at least two nurses as primary reviewers and one physician as secondary reviewer or mediator. The physician does not participate in the medical records review but authenticates the reviewers’ consensus and helps classify AE results by harm severity.

It should be noted that the main objective of GTT is to identify and classify the patient harm and not to determine the possibility of avoiding it.^17,18,22

This study was based on the administration of the GTT methodology developed by the IHI.²² The procedure was divided into two stages:

Stage 1 – Translation and cross-cultural adaptation, as recommended by Estrela²⁵ (translation, back-translation, adaptation, content validation by a focus group (FG), and pre-testing).

Stage 2 – Evaluation of the measurement properties of the translated instrument.

Procedure

Stage 1 – Translation and Cross-Cultural Adaptation

Phase I – The process of translating the GTT into Portuguese started with two independent translations of the original version by two bilingual APDH members, one very experienced in the subject under study and the other with no experience. Subsequently, together with a public health professor, experienced in the scale validation process, a consensus meeting was held, and the initial version was validated.

Phase II – Since the scale is simple and there were only residual differences between the independent translations, the initial version of the scale was back-translated into English by a native bilingual translator.

Phase III – To obtain semantic, conceptual and functional equivalence of the items, the initial version was analyzed by a focus group (FG) with 15 experts, including two internal medicine physicians, 11 nurses (five specialist nurses and six general care nurses), and two pharmacists, in a training context that included two moments: theoretical (2 hours) and practical (5 hours). The FG discussed the equivalence between the initial version and the original text and suggested semantic and functional changes to the GTT, which included adding two new triggers in the general care module (C15 – sleep disturbance, and C16 – skin lesions/maceration), and one in the medication module (M13 – Pain/blood pressure (BP)). A separation between types of infection and types of procedures was also recommended in trigger C11 (healthcare-related infection) and C14 (complication of a procedure). To be closer to the Portuguese reality, they recommended changing trigger M7 (diphenhydramine administration (Benadryl)) to “diphenhydramine, cetirizine dihydrochloride, hydroxyzine and clemastine administration”, and replacing the word BiPAP (bi-level positive airway pressure) by “Bi-Level” in trigger S4 (intubation or reintubation or use of BiPAP in the post-anesthetic care unit). As for the adaptation of triggers in the intensive care module, the FG agreed that it did not need changes; however, the group unanimously defined that the perinatal module should be eliminated, since it does not add value in the context of the medical-surgical department. In fact, the authors of the tool make it very clear that the modules should be used only when relevant, according to the type of care requested during hospitalization.¹⁸ As a result of the FG consensus, the preliminary Portuguese version GTT-PT was validated.

Phase IV – The FG conducted pre-testing in the practical moment of the training with the participation of two physicians and 11 nurses. Each nurse received five clinical processes to simulate a process review using the preliminary GTT-PT version. All participants received the same processes. Subsequently, the physicians authenticated the reviewers’ consensus and helped classify AEs by severity of the damage. The Fleiss’ Kappa test showed 0.76 agreement in the group, with significance = 0.001 and margin of error of 0.05.

Finally, to confirm the accuracy, relevance and effectiveness of the GTT-PT tool translation into Portuguese and to comply with the Cross-Cultural Adaptation Protocol (CCAP), the same FG was requested to respond to four statements (translation accuracy, easy administration, effectiveness in identifying AEs, and relevance of integrating it into risk management) using a five-point Likert scale, in which 1 – completely disagree, 2 – disagree, 3 – neither disagree nor agree, 4 – agree, and 5 – completely agree.

The Content Validity Coefficient (CVC) was calculated for each item using the final evaluation of these responses, which was excellent, with 0.98 for accuracy, 0.96 for easy administration, 0.96 for effectiveness in identifying AEs, and 1.0 for relevance of integrating GTT-PT in risk management.

Stage 2 – Evaluation of GTT-PT Measurement Properties

The measurement properties evaluation of GTT-PT included the analysis of a sample of 90 medical records randomized from a population of 581 hospitalizations between August 1, 2017 and April 30, 2018 using the Open Epi software.

The sample included complete medical records, with both medical and nursing notes, and clinical discharge records. It also included the concluded administrative procedures of patients discharged at least 2 months before and with a hospital stay longer than 24 hours. The files of patients under the age of 18, psychiatric patients in the acute phase, and with legibility issues were excluded from the sample.

The study included two nurses as primary reviewers and one physician to validate the consensus. Specific forms were developed to facilitate data collection. Independently, each reviewer analyzed the same medical records screening positive and negative triggers.

Nurses review took no more than 20 minutes for each admission. Ninety medical records were included in the study which relates to 176 admissions that were reviewed, since 46 (51.1%) had more than one admission episode during last year. Thirty-seven triggers from the original tool were used and three were added by the authors after the consensus of the FG.

Tool validity was determined by comparing AE found using GTT-PT triggers (T+) and AE found during process review without GTT-PT triggers (T−).

The data were recorded and processed using the IBM SPSS Statistics software for Windows, version 25.0. The Cronbach’s alpha was used to evaluate internal consistency. Interobserver agreement was evaluated using the intraclass correlation (ICC). Sensitivity, specificity, and accuracy were calculated as a diagnostic test for the GTT-PT. For precision, the positive (PPV) and negative predictive value (NPV) and the positive (+LR) and negative likelihood ratio (−LR) were calculated. Tool effectiveness was evaluated using relative risk (RR), absolute risk (AR), and odds ratio (OR). The prediction model was performed using multiple logistic regression. The discriminatory power of the model was evaluated through the analysis of the receiver operating characteristic (ROC) curve. The area under the curve (AUC) identified the potential presence of AE, being considered the gold standard.

Results

The process of cross-cultural adaptation of the IHI-GTT tool resulted in the GTT-PT version for hospital context, specifically the medical-surgical departments, with a substantial agreement of 0.76 through the Fleiss’ Kappa test and with excellent CVC (0.96–1.0).

The random sample used to assess the measurement properties of the GTT-PT consisted of 90 medical records, according to the established inclusion and exclusion criteria. A total of 176 admissions were analyzed, since 51.2% (46) of cases had more than one admission episode during last year.

The patients’ mean age was 77.02 years, with a standard deviation (SD) of 11.7 years, 52.2% being women. The mean number of days of the last admission was 20.5 days, and only 48.8% had a single admission.

The number of days in the last hospitalization was 1842, with a mean of 20.5, a median of 15.5 and SD of 17.2. On the other hand, the number of total days of hospitalization during last year was 3370, with a mean of 37.4, a median of 25 and SD of 35.8 (Table 1).

Table 1 Characteristics of the Patients in the Sample

The review of the included medical records identified 563 triggers. It was found that only 7 medical records did not present any triggers. It should be noted that 290 identified triggers were present in the records that presented only one admission and 273 in the records that presented between 2 and 5 admissions during last year. Due to the fact that all last year’s admissions were reviewed and examined regardless of the number of admissions per medical record, a larger number of triggers were found which allowed to identify the origins of AE.

After screening, 381 triggers were considered positive (T+) and could lead to AE identification. Negative triggers (T−) were considered as triggers that had no evidence of possible AEs or their presence was replicated throughout the review of the same medical record. After the consensus of the two reviewers, 142 AE were classified in total of admissions, 139 AE were found by the presence of the triggers and 3 AE were identified without trigger (Figure 1).

Figure 1 Description of the AE screening process.

It was found 42.1 AEs per 1000 patients/day, 157.8 AEs per 100 admissions and in 66.7% of hospitalized patients were targeted by at least 1 AE.

The prevalence of AE found was 36%. The study uses the number of triggers as a common denominator, as it aims to assess the ability of the instrument to identify AE.

The Cronbach’s alpha showed a global internal consistency of 0.83 for the GTT-PT at a level of significance of 5%. Also, the absolute agreement was validated through ICC to evaluate reliability, which proved to be excellent at 0.977 with a 95% confidence interval (CI) and significance = 0.001.

The psychometric values of the tool were obtained through sensitivity (97.8%) and specificity (74.8%), with a cutoff point of 0.5, and accuracy (83%), with PPV (69.8%), NPV (0.98%), +LR (3.88), and −LR (0.026).

Also, the GTT-PT effectiveness was measured using RR, AR, and OR (Table 2).

Table 2 The GTT-PT Global Predictive Properties

Multiple linear regression was used to understand which types of triggers can predict the most common AE. The analysis resulted in a statistically significant model for the predictors’ infection associated with health care, use of restrictions, number of hospitalization days, and sudden medication discontinuation (F (1.822) = 23.504, p < 0.002, adjusted R² = 0.278). The non-multicollinearity prerequisite of the predictive model was fulfilled (tolerance = 0.988, VIF (variance inflation factor) 1.012), and the index of independent residuals was acceptable through the Durban–Watson test (1.70). The dispersion graphs in Figure 2 show normal standardized residual distribution and normal linear distribution between dependent (gold standard) and independent variables.

Figure 2 Standardized residual regression graph (A) and dependent and independent variable dispersion graph (B).

The discriminatory power of the model and the tool accuracy were evaluated using the ROC curve resulting from GTT-PT sensitivity and specificity to detect AE (Figure 3).

Figure 3 ROC curve graph with the respective area under the curve (AUC).

ROC curve analysis shows an AUC of 0.872, with an inferior limit of 0.786 and an upper limit of 0.957 for significance p ≥ 0.001.

Discussion

The main objective of this study was to adapt and validate the GTT-PT tool to identify AE in the Portuguese context for hospital use in medical-surgical departments. The methodological process of validation and cultural adaptation of the GTT-PT followed the internationally recommended CCAP. The translation and back-translation stages presented insignificant differences, requiring slight changes, complying with the CCAP. The evaluation by the FG was essential to improve the Portuguese version, and the changes implemented increased the predictive value of the instrument. Three more triggers were added, and types of infection and complication were included in triggers C11 “healthcare-associated infection” and C14 “complications of a procedure”. The agreement of the FG in the pre-testing phase confirmed translation accuracy, easy administration, effectiveness in identifying AEs, and relevance of integrating GTT-PT in hospital risk management.

The high internal consistency obtained (Cronbach’s alpha = 0.83) showed that the GTT-PT was accurate. These data cannot be compared with other studies, since other publications on the GTT validation did not evaluate internal consistency^{23,24,26–29}; however, the IHI recommends to evaluate it.²¹

The reproducibility of the tool was confirmed by an ICC of 0.977 (CI = 95%, p > 0.001), considered excellent in the absolute inter-reviewer agreement. German,²⁹ British,¹⁸ and Norwegian³⁰ studies also evaluated agreement using Cohen’s Kappa coefficient, which was 0.7–0.8, 0.81, and 0.61, respectively. Apparently, these differences are associated with primary reviewer training.^26,30

The GTT-PT identified an AE prevalence of 36%, similarly to the values reported by Brösterhaus²⁹ (32.5%), Mevik¹⁷ (34.7%), and Parrinello²⁶ (30.4%). However, Toscano²⁴ and Karpov²⁷ reported much lower values (24.7% and 13.2%, respectively). This difference is due to the different patients and contexts analyzed.

As for measurement properties values, sensitivity was 97.8% and specificity 74.8% at a cutoff point of 0.5 and 83% accuracy, with VPP 69.8% and VPN 0.98%. The calculation +LR showed that the T+ clinical processes presented 3.88 times more probability of finding an AE, and that in the T− processes, this probability was 0.026 (−LR). The presence (AE+) or absence (AE−) of AE constituted the gold standard of the tool.

Similar results were obtained by Karpov (sensitivity 99.3%, PPV 57.1–100, and NPV 87.1–88.2).²⁷ Zimlichman²³ reports that the Israeli GTT version presented a sensitivity of 97%, but a PPV of 17.8%, which was lower than in the GTT-PT version, and Toscano²⁵ reported only a PPV of 19.6% and NPV of 65.5%.

In addition, the results of this study show that T+ processes were 138.2 times more likely (OR) to find an AE than the T– processes, and that the AE+ group of patients with T+ presented an RR risk equivalent to 42.6% of the risk found for patients with T–. As for AR, 0.68 patients could have an additional risk of having an AE even if there was no trigger.

The discriminatory power of the GTT-PT model analyzed through the ROC curve showed the excellent performance of the tool to identify AE (AUC = 0.872) for p ≥ 0.001.

The comparison of these data with data from other studies was not possible because they did not analyze measurement properties in the diagnostic evaluation of the tool. Only the study by Parrinelo²⁶ reported that the ROC curve showed that the probability of an AE is higher in medical records with two more triggers.

Multiple linear regression showed that the most frequent triggers predicting the most common AEs were healthcare-associated infection, use of restrictions, number of hospitalization days, complications of a procedure, and sudden medication discontinuation. The most used triggers were from the general care and medication module, corroborating the German,²⁹ Chinese²⁸ and Sicilian²⁶ studies. The Spanish study²⁴ reports that prolonged hospitalization was an AE predictor, and Zimlichman²⁴ specified that sudden medication discontinuation was an AE predictor.

Although different values were found in other studies analyzed and considering the psychometric capabilities of the GTT-PT, this tool was considered valid, sensitive, and reproducible to identify AEs in the medical-surgical departments, being potentially more adequate compared to other traditional methods of AE identification.^18,21 As it is a simple administration tool, it continuously shows the real magnitude of the context analyzed allowing rapid intervention of hospital risk management, thus guaranteeing PS. Therefore, the implementation of new tools to identify AEs that are valid and adapted to the Portuguese context becomes essential, thus expanding the systematic implementation of continuous improvement in early AE prevention.

The limitations of the present study are not only the scarcity of international scientific production related to the theme, making it difficult to compare the data with other realities, but also the fact that most existing studies focus only on the evaluation of prevalence, sensitivity, and PPV.

Therefore, consistently with scientific evidence²⁵ to improve the quality and reliability of instrument validation studies, these studies should evaluate the internal consistency, reliability, and reproducibility of the tools.

This study described the GTT-PT validation and cultural adaptation process, according to the methodological rigor described in the international literature,²⁵ maintained the measurement properties as recommended by the IHI.²²

Conclusion

The adapted version was considered adequate regarding semantic, idiomatic, cultural, and conceptual equivalences, being approved by the authors of the original instrument.

The GTT-PT version presented good measurement properties quality, with excellent predictive values and high internal consistency, reliability, and reproducibility rates.

The translated and adapted GTT-PT is an important instrument to identify AEs in the Portuguese hospital context in the medical-surgical departments. Therefore, the present tool improves knowledge on the real AE magnitude and impact. Unquestionably, recognizing AEs before the implementation of measures and programs to continuously improve risk management is a fundamental requirement for the subsequent reduction of material costs, financial losses, and social impact.

The application of the tool in other national services could also contribute to the robustness of GTT-PT in medical and surgical departments. Further studies on the measurement properties evaluation of the GTT will be able to evaluate other properties, as well as its behavior in other populations.

The GTT-PT translation, adaptation and validation was an important step in the area of PS, providing health institutions with another tool to diagnose and analyze AEs.