The Quality and Clinical Applicability of Recommendations in Ostomy Guidelines: A Systematic Review

Plain Language Summary

Impact

• What problem did the study address?

Despite rapid development of ostomy guidelines in a wide range of institutions and locations, the diversity of settings, objectives, intended users, inclusion criteria, and methods for collecting and analyzing evidence are linked to differences in the methodological quality of clinical practice guidelines. The current review assessed the quality and clinical applicability of ostomy guidelines.

• What were the main findings?

Although the quality of ostomy guidelines varied, we identified three high-quality guidelines. Most ostomy guidelines gave little consideration to applicability or to the values and preferences of patients, policymakers and intended users. The development of more high-quality guidelines and more clinical applicability guidelines is required. Ostomy-related complications, especially parastomal hernias, are addressed by most ostomy guidelines. Pediatric ostomy care needs more research attention. Intended user of guidelines for patient needs to be developed.

• Where and on whom will the research have impact?

This review will help guideline developers to improve the quality of ostomy guidelines in the future, and help ostomy nurses/clinicians select and implement better ostomy clinical practice guidelines.

Introduction

An ostomy is a surgically constructed abdominal wall opening that allows feces and urine to be drained outside the body.¹ Internationally, 1.3 million individuals are estimated to have an ostomy.² The construction of an ostomy is a life-altering event that affects many areas of health-related quality of life.³ Despite the many challenges involved, it is feasible for people to have active and fulfilling lives following ostomy surgery if the necessary services and assistance are available. To standardize care and to help practitioners make healthcare decisions for the management of medical concerns, many clinical practice guidelines (CPGs) have been developed in recent years. Although implementing the recommendations of ostomy CPGs can help to decrease stoma-related complications and enhance overall health-related quality of life,^4–6 not all CPGs have rigorous methodologies. Low-quality CPGs can result in ineffective or even harmful practices and poor patient outcomes. However, no comprehensive and rigorous evaluations of ostomy CPGs and their recommendations have been conducted. The Appraisal of Guidelines for Research and Evaluation II (AGREE II)⁷ and the AGREE Recommendation EXcellence (AGREE-REX)⁸ are methodological tools for assessing the quality of guidelines and clinical applicability of recommendations, and are commonly used internationally. Therefore, it is necessary to evaluate ostomy CPGs using these two tools to help ostomy nurses/clinicians select and implement better ostomy CPGs. In addition, it is necessary to provide guidance for the development of higher-quality guidelines in the future.

Background

Ostomy is often performed in patients with colorectal cancer, bladder cancer, bowel obstruction, and other etiologies.⁹ The ostomy types mainly includes colostomy, ileostomy, and urostomy,¹ which refer to a surgically created opening from the colon, small intestine, or isolated segment of the ileum to the abdominal wall to allow the elimination of feces or urine.¹⁰ Living with an ostomy can have adverse impacts on body image, sexual function, mood, and everyday activities.^11,12 Stoma-related complications are one of the most common physical consequences of having a stoma.¹³ People undergoing ostomy surgery, whether temporary or permanent, face numerous challenges and lifestyle changes.¹⁴

CPGs are statements that include recommendations based on a systematic review of evidence and an assessment of the benefits and harms of different care alternatives.¹⁵ Ostomy CPGs can provide nurses/clinicians with various evidence-based recommendations to support people who are planning to have an ostomy or who already have an ostomy. These recommendations include preoperative and postoperative care needs such as stoma site marking, education, and follow-up.¹ Most CPGs can apply to colostomy, ileostomy, and urostomy.¹ Implementing the recommendations of CPGs has been reported to decrease complications and improve quality of life.^4–6 Although multiple ostomy guidelines exist, the choice of which guidelines to implement can be difficult because not all CPGs have a high level of quality.

At present, tools for evaluating the quality of guidelines include AGREE II,⁷ AGREE-REX,⁸ AGREE-China,¹⁶ and Reporting Items for Practice Guidelines in Health Care (RIGHT).¹⁷ The AGREE II is a methodological tool for evaluation, formulation, and reporting of guidelines.⁷ This tool comprises 23 items arranged into six domains (scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence).⁷ AGREE II is widely used to appraise the overall methodological quality of CPGs.¹⁸ However, a high AGREE II score does not ensure that the recommendations of guidelines are clinically credible and implementable.⁸ For instance, AGREE II was used to appraise official disability guidelines, and achieved a high score (eg, the clarity of the presentation domain score was 75%; scale: 0–100%).¹⁹ However, expert clinicians reported that the recommendations have less utility in the non-guideline development setting. In addition, for several themes, the content was of doubtful validity. Therefore, the AGREE-REX tool was established as a companion of the AGREE II tool, to appraise the clinical credibility, trustworthiness, and implementability of recommendations.⁸ The AGREE-REX has nine items representing three domains (clinical applicability, values and preferences, implementability).⁸

To meet the needs of Chinese guideline evaluation, AGREE-China was developed on the basis of the AGREE II.¹⁶ AGREE-China has 15 items and five checkpoints (science and rigor, effectiveness and safety, economy, usability, and feasibility, and conflicts of interest). RIGHT was developed to address the gap between poor quality of reporting practice guidelines and the lack of widely accepted standards for reporting in health care.¹⁷ This instrument includes 22 items that are considered to be essential for good reporting of practice guidelines: basic information, background, evidence, recommendations, review and quality assurance, funding and declaration and management of interests, and other information.¹⁷ Compared with AGREE II, RIGHT places more emphasis on the guidance of guideline reporting.¹⁷

To help nurses/clinicians select the most appropriate ostomy guidelines to implement, it is necessary to assess the quality of guidelines and clinical applicability of recommendations. However, no comprehensive and rigorous evaluation of ostomy CPGs and their recommendations is currently available.

The Review

Aim

The current review sought to systematically search ostomy CPGs, critically assess the quality and applicability of the recommendations using the AGREE II and AGREE-REX, and summarize these recommendations to improve future ostomy guideline development and help ostomy nurses/clinicians select and implement better ostomy CPGs.

Design

A systematic review was conducted and reported following the Reporting Items for Systematic Review and Meta-Analyses guidelines. Prior to conducting the review, we filed the protocol with the international prospective register of systematic reviews (registration number: CRD42022302875).

Search Methods

Search Strategies

We developed search strategies with the help of an enterostomal therapist and an experienced senior librarian. Electronic databases, guideline repositories, and ostomy institution websites were searched. Electronic databases include PubMed, ProQuest, and CINAHL (EBSCO interface). The guideline repositories included the Registered Nurses’ Association of Ontario (RNAO), the National Institute for Health and Care Excellence (NICE), CPG Infobase: Clinical Practice Guidelines (Canadian Medical Association), the BMJ Best Practice and Clinical Key (Elsevier) and four other repositories. Ostomy institution websites included the World Council of Enterostomal Therapists (WCET), the Wound, Ostomy and Continence Nurses Society (WOCN), and the United Ostomy Associations of America (UOAA).

The search strategies were tailored to the characteristics of each database, repositories and website using the following medical subject headings (MeSH) and keywords as search terms: “ostomy”, “stoma”, and “guideline.” This review focused on guidelines published between 2011 and 2021. The initial search was conducted on June 2, 2021, then updated on September 3, 2021. An example PubMed search is as follows: ((“ostomy” [Mesh]) OR (“ostomy”[TI/AB]) OR (“stoma”[TI/AB])) AND ((“guideline”[Mesh]) OR (“guideline*” [TI/AB])).

Eligibility Criteria

The following criteria were used to determine guideline eligibility. Inclusion criteria: recommendations about ostomy care and management, with an explicit statement that it is a guideline; colostomy, ileostomy and urostomy guidelines; temporary and (or) permanent ostomy; English-language guidelines; and the latest version of guidelines. Exclusion criteria: incomplete guidelines (eg, abridged versions); lack of certain components that should be included in a guideline, such as a structured evidence review; best practice, the practice of policy statements, description of guideline development, practical guides; an absence of specific recommendations in the guidelines for ostomy care and management, the guidelines only mention ostomy (eg, Bladder cancer: diagnosis and management).

Process of Including and Excluding Articles

All search results were imported into Endnote (version X9). Two authors (L, Z) independently screened titles and abstracts after eliminating duplicates. Without abstracts, the title was used to assess eligibility. For relevant records, full-text versions of manuscripts were acquired and screened. Guidelines were included and excluded according to the eligibility criteria. Disparities in these processes were resolved with the help of the third author (P).

Search Outcomes

Electronic databases, guideline repositories, and ostomy institution websites yielded 1475 records, and 183 duplicates were then deleted. 27 full-text articles were obtained and screened, and 17 were excluded. Finally, 10 ostomy CPGs were included (Figure 1).

Figure 1 The flow diagram of studies selection.

Quality Appraisal

AGREE II was used to appraise the overall methodological quality of each included CPG. Each item of this instrument is ranked on a seven-point scale (1 [strongly disagree] to 7 [strongly agree]).⁷ The domain scores are computed by adding all of the individual item scores, then standardized to the maximum possible score. The calculation formula is as follows: (obtained score – minimum possible score)/(maximum possible score – minimum possible score) × 100%. Each domain has a standardized score ranging from 0% to 100%.

AGREE-REX was used to appraise the credible and implementable recommendations. Similar to AGREE II, each item is graded on a seven-point scale, and the domain score can be standardized.⁸ More detail related to the AGREE II and AGREE-REX tools is available on the AGREE research trust website (https://www.agreetrust.org).

The average score was calculated for each guideline: ((the guideline AGREE II average scores) + (the guideline AGREE-REX average scores))/2. Guidelines were then ranked on the basis of these average scores. The top three were ranked as high, the middle three were ranked as moderate, and the bottom four were ranked as low.

The quality of the 10 included guidelines was appraised independently by two authors (L, Z). Before the appraisal, two authors used the online training tool in the My AGREE PLUS platform (https://www.agreetrust.org) to ensure proficiency and accuracy in using the AGREE II tool. The reliability of the score assigned by the two reviews was evaluated using intraclass correlation coefficients (ICC). All of the ICC values were > 0.75. The two reviewers used a consensus approach to reach agreement about the AGREE-REX item scores. Before appraisal using the AGREE-REX tool, two reviewers evaluated two CPGs in advance (no online training was available for the AGREE-REX tool).

Data Extraction

The characteristics (eg, title, author, year, and objective) of the 10 included guidelines were extracted by one author (L) using the developed extraction tool.²⁰ In addition, these data were confirmed by another author (Z). The methodological quality and implementability of the recommendations of the 10 included guidelines were presented through the six domain scores of the AGREE II and the three domain scores of the AGREE-REX. A summary of the 10 included guidelines recommendations and their level of evidence is given.

Results

Characteristics of Included CPGs

Table 1 shows the characteristics of the 10 included guidelines. In this review, the included CPGs were released between 2015 and 2021. The purpose of almost all of the included guidelines was to provide recommendations for the care and management of adult patients with stomas. The intended users of the guidelines included nurses, educators, leaders, policymakers, and researchers.

Table 1 Included CPGs Characteristics

Quality of Included CPGs

Quality of Each Included CPG

Table 2 shows the domain, total, and average scores of each included CPG using AGREE II and AGREE-REX.

Table 2 Each CPGs Domain Scores According to AGREEII and AGREE-REX

AGREE II: among the 10 guidelines, 2019 RNAO¹ exhibited the highest total scores, and the 2015 American Society of Colon and Rectal Surgeons (ASCRS)²¹ had the lowest total scores. 2019 RNAO¹ had the highest domain scores in four domains (“scope and purpose”, “rigor of development”, “applicability”, and “editorial independence”). 2020 WCET²² had the highest domain scores in two domains (“stakeholder involvement” and “clarity of presentation”). In the “editorial independence” domain, 2018 WOCN¹⁴ had the highest score. In the “rigor of development” domain, 2018 EHS²³ had the highest score.

AGREE-REX: 2019 RNAO¹ had the highest total score and highest domain scores in all three domains. 2015 ASCRS²¹ had the lowest total score.

AGREE II and AGREE-REX: The “high” quality guidelines were 2019 RNAO,¹ 2019 NICE,²⁴ and 2018 EHS.²³ The “moderate” quality guidelines were 2020 WCET,²² 2018 WOCN,¹⁴ and 2019 MISSOTO.²⁵ The “low” quality guidelines were 2021 MISSOTO,²⁶ 2016 Toronto,²⁷ 2018 CAET,²⁸ and 2015 ASCRS.²¹

Quality of Each Domain of All Included CPGs

AGREE II: Figure 2 shows each AGREE II domain score of all included CPGs. The median of the “editorial independence” domain was the highest (86%), while the median of the “applicability” domain was the lowest (45%). The “clarity of presentation” domain was most concentrated in relation to scores within a range of 72% to 94%. The “editorial independence” domain was the most decentralized in relation to scores within a range of 4% to 100%.

Figure 2 AGREE II domains scores of all included CPGs.

AGREE-REX: Figure 3 shows each AGREE-REX domain scores of all included CPGs. The median of the “clinical applicability” domain was the highest (72%), while the median of the “values and preferences” domain was the lowest (38%). The “clinical applicability” domain was the most concentrated in relation to scores within a range of 50% to 94%.

Figure 3 AGREE-REX domains scores of all included CPGs.

Recommendations and Levels of Evidence

Table 3 shows the recommendations and their levels of evidence from the included CPGs. The 10 CPGs contain 172 recommendations in total. Most of the recommendations are based on the evidence’s veracity across the included guidelines, which was found to be relatively consistent.

Table 3 Summary of Recommendations for Ostomy and Their Levels of Evidence

Discussion

To the best of our knowledge, the current study is the first systematic review to evaluate the quality and clinical applicability of ostomy CPG. The 10 included CPGs were of relatively high-quality in the majority of the AGREE II and AGREE-REX domains (Table 2). The 2019 RNAO guideline¹ achieved the highest total scores using the AGREE II and AGREE-REX tools. Two other guidelines (2019 NICE,²⁴ 2018 EHS²³) were also ranked as “high” quality, and evaluated as “recommended.” Thus, the current results indicate that three high-quality guidelines may be appropriate first choices for nurses or physicians when performing ostomy care and management.

The results revealed discrepancies in methodological quality between CPGs, particularly in the applicability (AGREE II domain) and values and preferences (AGREE-REX domain) domains (Figures 2 and 3). These findings suggest that the quality regarding stoma guidelines and clinical applicability of recommendations should be further improved. The current results are consistent with quality appraisal results of previous studies in other clinical areas, including pressure injury CPGs¹⁸ and melanoma CPGs.²⁹

The median “applicability” was lowest among the six AGREE II domains (Figure 2). The median applicability score reported in a previous study was 42% across 137 guidelines published in 2008 or later.³⁰ This is similar to the median observed in the current review. Limited application of guidelines can lead to the omission of good healthcare, avoidable injury, poor patient outcomes or experiences, and resource waste.³¹ It is important to promote guideline applicability. Additional resources may be required to implement the recommendations. More specialist personnel, new technology, and costly drug treatment, for instance, may be required. These factors could have a financial impact on healthcare costs.⁷ However, almost all of the guidelines lacked explicit information about cost in this review. Some studies have focused on how to incorporate resource and cost evidence when formulating recommendations.³² Importantly, consideration of the potential resource impact of recommendations should also be undertaken by guideline developers.

Most included guidelines were insufficient in the “values and preferences (AGREE-REX domain)” of target users, public/patients, and policymakers (Figure 3). Previous research indicates that the importance of addressing the values and preferences of stakeholders in reliable guideline formulations is widely accepted.³³ Stakeholder involvement can also assist in maximizing the acceptance and feasibility of a guideline from the perspectives of intended users.³⁴ A notable exception is the 2019 RNAO guideline.¹ This guideline specifies the role of an expert panel in determining the scope of the guidelines and formulating questions to guide the literature review through focus group interviews. Nurses, clinicians, teachers, researchers, and policymakers made up this expert panel, which also included people who had ostomy experience and were interprofessional in composition. The expert panel also provided feedback on the draft of the guideline. Therefore, when developing guidelines, these methods of stakeholder involvement and engagement should be considered.

The recommendations of the included guidelines were consistent, although they used different evidence grading systems to indicate the levels of evidence (Table 3). All of the guidelines included in this study were found to include low to high levels of evidence that informed their recommendations. Some recommendations were made on the basis of low- or very low-level evidence, or even on the basis of experts’ opinions, representing uncertain clinical effects. More studies providing high-level evidence (eg, randomized controlled trials) regarding ostomy care should be conducted to support guideline developers in making recommendations.³⁵ The current results revealed that parastomal hernia prevention and management received the most attention among all recommendation subcategories. One potential reason for this finding is that parastomal hernia is the most common ostomy-related complication in various types of stomas.³⁶ Although parastomal hernias received extensive attention in the included guidelines, some surgeons do not believe that the currently available evidence is sufficient.³⁷ In addition, a study from Poland revealed low compliance with guideline recommendations for the repair of parastomal hernias.³⁸ Thus, there is a need for additional studies, high-quality evidence, and guideline application on this subject in future. This is even more important for other topics.

Most of the CPGs included in the current review focused on adults who have or will have an ostomy (Table 1). A lack of guidelines for pediatric ostomy patients was identified through this review. To address this shortcoming, a distinct search for evidence and methods for guideline creation was required. The intended users of most CPGs are nurses, educators, leaders, policymakers, and researchers, rather than patients. An ostomy is a long-term condition, and self-care enhances health in people with these conditions.³⁹ The current guidelines are very specialized and may not be suitable for patients. There is an urgent need to consider patients’ needs and health concerns as the central priority, and to build patient guidelines on the basis of evidence.

Limitations

The search strategies in the current review were designed with the help of an enterostomal therapist and an experienced librarian. This ensured that the search strategies were professional, comprehensive, and effective. However, some relevant CPGs were likely to have been overlooked because of language or publication restrictions.

Conclusion

The findings of the current review revealed that the quality and clinical applicability of included guidelines varied substantially. Three high-quality clinical practice guidelines, however, were identified. These guidelines appear to be suitable for implementing in daily practice, and can be tailored to the needs of the local context. The current results also suggested a need for greater promotion of applicability, values and preferences in developing ostomy guidelines. Consideration of the potential resource impacts of recommendations should also be undertaken by guideline developers to improve application. To improve values and preferences, stakeholder involvement need to be considered for formulating recommendations. There is a need for future studies and high-quality evidence to support the further development of ostomy guidelines. Pediatric ostomy care needs more research attention. Intended user of guidelines for patient needs to be developed.