Agreement of triage decisions between gastroenterologists and nurses in a hospital endoscopy unit

Introduction

Gastrointestinal endoscopy is a limited resource with significant direct and indirect costs¹ as well as a small but real risk of complications such as bleeding and perforation.² For these reasons, the indications for endoscopy are clearly defined by various organizations.³ The efficient and accurate triage of endoscopy referrals is essential to the functioning of an endoscopy service. It is time-consuming however, and when conducted by the endoscopists themselves reduces capacity for endoscopy. It is also an area of potential clinical risk as inappropriate triage can delay treatment and impact care. Careful and efficient triage is needed to ensure that the patients who are in most need of endoscopy have access in the timeliest manner.

Many triage decisions can be directly based on national and local triage criteria. While the information supplied by referrers does not always allow triage against these criteria, where the information is sufficient and the clinical situation matches the agreed criteria, the decision making for triage can be straightforward. We hypothesized that standardizing this approach and delegating it to expert endoscopy nurses could reduce the triage workload for the triaging endoscopist while maintaining the quality and consistency of triage decision making.

A PubMed search for the terms “nurse”, “triage” and “endoscopy” revealed no articles investigating the use of nurses for triage of endoscopy referrals. The literature regarding nurse triage mostly focuses on emergency medicine. Acceptable, but not perfect, inter-rater reliability between nurses has been seen in meta-analysis.^4,5 The studies that have examined nurse triage and compared it to physician triage have been emergency department-based studies performing triage on acute patients physically present in the department.^6,7 No studies were found that directly compared nurse triage of paper or electronic referrals with physician triage according to agreed criteria.

In New Zealand, national triage criteria for colonoscopy have been set by a working party of the Ministry of Health, and it is expected that public hospital endoscopy units will apply and adhere to these criteria in triaging referrals for colonoscopy (Supplementary materials).⁸ No such national criteria exist for gastroscopy, but Hutt Valley DHB utilizes its own, locally agreed, criteria (Supplementary materials [Hutt Valley DHB Gastroscopy Access Guidelines]). From these national and local criteria, triage algorithms for gastroscopy and colonoscopy referral were developed (Supplementary materials [colonoscopy triage decision model]).

This study aimed to provide data that would inform the design of a nurse triage service in endoscopy, determining the proportion of referrals that could be triaged by a nurse and the inter-rater reliability using a triage algorithm based on nationally and locally agreed triage criteria.

Materials and methods

This was a single-center prospective study. Hutt Hospital Endoscopy Department is a secondary care, university-affiliated endoscopy unit. A convenience sample of consecutive referrals to the department during a 4-month study period was chosen. Undifferentiated referrals to the department were assessed, including referrals for consultation rather than endoscopy. These referrals were included in order to assess the proportion of total referrals to the department amenable to nurse triage.

All referrals to the department were processed via a central electronic system; this system combines both referrals conveyed to the hospital by electronic means and scanned copies of paper referrals. All referrals were triaged by a consultant gastroenterologist using national and local triage criteria. The national criteria are published by the New Zealand Ministry of Health and are a consensus statement developed by a working party of the Ministry.⁸ The local triage criteria for gastroscopy were developed by clinicians at the Hutt Hospital. The triaging gastroenterologist applies these criteria to the referral whenever the information provided allows. However, they may make decisions based on their professional experience and knowledge where the referral falls outside these criteria. The patients were managed according to the recommendation of the gastroenterologist’s triage assessment as they usually would be.

Copies of the referral information were anonymized by administrative staff not involved in the study and then given to two endoscopy nurses. Each nurse was experienced in endoscopy, having >5 years experience working in endoscopy, but were not nurse endoscopists. They were asked to independently apply a clearly defined triage algorithm (Supplementary materials [colonoscopy triage decision model]) based on the national colonoscopy referral criteria and the local gastroscopy referral criteria. This triage algorithm was developed by a gastroenterologist experienced in triage who applied the existing referral criteria already in use into simple decision algorithms, one for colonoscopy and another for gastroscopy. The nurses had been provided with familiarization training for the use of the algorithm but had not been given any other formal training in triage. They were asked to provide a triage assessment for each referral only when the referral information and the algorithm permitted it. Where triage to endoscopy was not possible, a code of “unable to be triaged to endoscopy” was given. Each clinician was blinded to the triage category applied by the other clinicians.

Any referral could potentially be triaged to colonoscopy, gastroscopy, both colonoscopy and gastroscopy, or “unable to be triaged to endoscopy”. For colonoscopy, the triage priority could be A (most urgent, endoscopy to be performed with 2 weeks) or AB (least urgent, endoscopy within 6 weeks). For gastroscopy a further, lower priority, triage category of B was available (endoscopy outside of 6 weeks).

A panel of one gastroenterologist and one senior nurse working together, each of whom had >5 years experience in triage decision making for endoscopy, provided adjudication with reference to the existing referral criteria and the triage algorithm developed for use by the nurses. This panel was blinded to the triage category given by the study clinicians. The panel did not consider cases where all three clinicians agreed on the triage category (or inability to triage to endoscopy) but did give a triage decision for all remaining cases.

The protocol met the definition of a minimal risk observational study involving only anonymized health information and as such was exempt from review as per the New Zealand Health and Disability Ethics Committee’s standard operating procedures.¹⁰

Statistical methods

The inter-rater agreement between the two nurses and the gastroenterologist was measured by Cohen’s kappa coefficient as was the inter-rater reliability between the individual clinicians and the adjudication panel in those cases where the three clinicians did not assign identical triage codes. The proportion of patients for whom the nurses were able to assign a triage category was also recorded.

All data were entered in an Excel spreadsheet (Microsoft Corporation, Richmond, WA, USA) and analyzed with SPSS (version 23; IBM Corporation, Armonk, NY, USA). We used Cohen’s kappa statistic to measure the agreement between pairs of assessors. This statistic takes into effect the percentages of agreement that would be expected by chance. Possible values for the kappa statistic are from –1 to 1, with 1=perfect agreement, 0=completely random agreement and –1=perfect disagreement. Based on the study of Landis and Koch,we can interpret values between 0.0 and 0.2 to indicate slight agreement, 0.21 and 0.40 to indicate fair agreement, 0.41 and 0.60 to indicate moderate agreement, 0.61 and 0.80 to indicate substantial agreement and 0.81 and 1.0 to indicate almost perfect/perfect agreement.⁹

No preexisting data were available to guide sample size assessment, and a convenience sample of 105 referrals was taken.

Results

A total of 105 consecutive referrals triaged in the department between January 30, 2015 and April 17, 2015 were considered. Of these unselected referrals, the gastroenterologist was able to triage 64 (61%) to endoscopy, Nurse A was able to triage 54 (51%), and Nurse B was able to triage 44 (42%). In 38 (36%) cases, all three agreed that the referral could not be triaged to endoscopy.

The results for the comparisons between clinicians are summarized in Table 1. For 40 (38%) referrals, all three were able to assign an endoscopy triage code. For a total of 21 (20%) referrals, an endoscopy triage code was given and there was agreement between all three clinicians.

Table 1 Kappa values for inter-rater agreement according to the triage code assigned Notes: aSubstantial, bmoderate and cslight agreement. n, sample size for the comparison.

To determine the level of agreement between clinicians as to the triage code assigned, Cohen’s κ was run. The agreement between the two nurses was substantial, and between the gastroenterologist and nurses, moderate agreement was seen.

For the 46 (44%) referrals where agreement between the three clinicians was not absolute, an adjudication panel made up of a consultant gastroenterologist and senior endoscopy nurse, both experienced in triage, was asked to assign a triage category where possible or state if they believed it was not possible to triage to endoscopy. They were asked to make a consensus decision based on their experience and by referring directly to the local and national triage criteria and the triage algorithm used by the nurses.

Regarding the triage code assigned, moderate agreement was seen between the nurses and this adjudication panel. However, there was only slight agreement between the adjudication panel and the gastroenterologist.

Discussion

Our study is the first to demonstrate that nurse triage of undifferentiated gastroenterology referrals using a decision algorithm is feasible. Of all undifferentiated referrals, our study nurses were able to apply an endoscopy triage code to between 42% and 51% of referrals. This was compared to 61% of referrals being able to be triaged to endoscopy by our gastroenterologist.

This approach could lend itself to computer-assisted triage in the future. However, the organizational changes needed to achieve such a system are some way off in our setting. Referrals come from multiple sources and are of varying complexity, completeness and quality. Currently, our systems rely on a human being to interpret and assign a triage category to referrals.

This study was limited by the lack of a gold standard against which to compare clinicians. Such a study could be conducted using “test” referrals written specifically to assess the accuracy of the clinician performing triage. This approach would also allow an analysis regarding the types of referral where triage clinicians performed well and not so well. We elected to use real-life sequential referrals as we were interested in the proportion of referrals a nurse would be able to triage to endoscopy in our service and the accuracy of triage. This study is unlikely to be applicable in diverse clinical situations because of differences between triage recommendations and service organization. However, it does demonstrate the principal that nurses can triage a substantial proportion of unselected gastroenterology referrals to endoscopy.

It is possible that the accuracy of triage and the proportion of referrals able to be triaged by nurses could be improved by more formal training. However, we wished to test a pragmatic approach where registered nurses working in endoscopy, who were not nurse endoscopists, could be asked to perform triage tasks with limited training and a robust decision algorithm.

In different countries, the cost-benefit of this approach would vary depending on relative remuneration. We did not conduct a time in motion study to show the relative time efficiency of the clinicians, and for this reason, no cost-benefit analysis was undertaken. Further studies of this approach should be designed to address this point.

Conclusion

This study confirms that triage is a highly user-dependent activity but supports the hypothesis that standardizing the triage approach and delegating it to expert endoscopy nurses could reduce the triage workload for the triaging endoscopist while maintaining the quality and consistency of triage decision making. Nurse triage using a decision algorithm developed from local and national guidelines can approximate decision making by an experienced clinician using the same guidelines in clinical practice. Any triage strategy should involve multiple checks and balances and might benefit from application of formal decision tools.

Contribution of this article

What is already known about the topic?

• Nurse triage is reliable and effective in ED triage and primary care.
• Triage of endoscopy referrals is resource intensive and a potential source of clinical risk.

What this paper adds?

• This paper shows that nurses can reliably triage undifferentiated gastroenterology referrals to endoscopy with substantial agreement between nurses and moderate agreement with an experienced triage gastroenterologist and an adjudication panel.

Acknowledgment

Mrs Rosemarie Sandford RN contributed to the design and data collection for this study.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Sharara N, Adam V, Crott R, Barkun AN. The costs of colonoscopy in a Canadian hospital using a microcosting approach. Can J Gastroenterol. 2008;22(6):565–570.
2.		Dominitz JA, Eisen GM, Baron TH, et al. Standards of Practice Committee. American Society for Gastrointestinal E: Complications of colonoscopy. Gastrointest Endosc. 2003;57:441–445.
3.		ASGE Standards of Practice Committee, Early DS, Ben-Menachem T, et al. Appropriate use of GI endoscopy. Gastrointest Endosc. 2012;75(6):1127–1131.
4.		Mirhaghi A, Heydari A, Mazlom R, Ebrahimi M. The Reliability of the Canadian Triage and Acuity Scale: Meta-analysis. N Am J Med Sci. 2015;7(7):299–305.
5.		Parenti N, Reggiani ML, Iannone P, Percudani D, Dowding D. A systematic review on the validity and reliability of an emergency department triage scale, the Manchester Triage System. Int J Nurs Stud. 2014;51(7):1062–1069.
6.		Nissen L, Kirkegaard H, Perez N, Hørlyk U, Larsen LP. Inter-rater agreement of the triage system RETTS-HEV. Eur J Emerg Med. 2014;21(1):37–41.
7.		Buschhorn HM, Strout TD, Sholl JM, Baumann MR. Emergency medical services triage using the emergency severity index: is it reliable and valid? J Emerg Nurs. 2013;39(5):e55–e63.
8.		Referral Criteria for Direct Access Outpatient Colonoscopy or CT Colonography. New Zealand Ministry of Health. 2015;2015.
9.		Operating procedures. New Zealand Health and Disability Ethics Committees. 2015;2012.
10.		Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174.