Evaluation of Guidelines and Consensus on Ectopic Pregnancy Based by AGREE II Method

Introduction

Ectopic pregnancies (EPs) represent a severe early pregnancy complication which means implantation of a developing blastocyst that occurs outside the endometrial cavity of the uterus.¹ It is associated with increased risks of maternal morbidity and mortality. In order to optimize the diagnosis, treatment, and management of ectopic pregnancy, several clinical guidelines have been published to guide clinical decisions.

Clinical practice guidelines (CPG) are designed to help doctors make appropriate clinical decisions based on current evidence.² Different guidelines are generally developed by different organizations to be applicable to the corresponding regions. The potential value of guidelines mainly depends on their quality, and high-quality guidelines are able to provide clinical decisions with a high evidence-based level. The quality of the guidelines may vary due to differences in development time, methods, and so on. Therefore, it is necessary to conduct a comprehensive evaluation of the quality of existing ectopic pregnancy guidelines to assess their applicability in clinical practice.

The Appraisal of Guidelines for Research and Evaluation (AGREE) instrument is used to assess methodological quality of guidelines in six domains: scope and purpose, participants, rigor, clarity of presentation, applicability, and editorial independence.³ The purpose of this study was to evaluate the quality of ectopic pregnancy guidelines in order to provide reference for the development of ectopic pregnancy guidelines.

Methods

We searched China National Knowledge Infrastructure (CNKI), Wanfang Med, PubMed database, Embase database, National Guideline Clearinghouse (NGC), and the National Institute for Health and Clinical Excellence (NICE) in the UK. The publication time was from 2010 to 2022, and relevant references were manually added. The search terms included “ectopic pregnancy”, “guideline”, “expert consensus”, “recommendation”, “opinion”.

Inclusion criteria were as follows: (1) English or Chinese language, (2) based on systematic evidence synthesis and containing specific statements that guide ectopic pregnancy decisions, (3) developed by professional organization(s) for the diagnosis and management of ectopic pregnancy, (4) published between 2010 and 2022, only the latest editions were included.

Exclusion criteria were as follows: (1) Previous editions published by the same academic organization, (2) lectures or expert reviews, (3) review or research literature, (4) guide interpretation, and (5) guidelines that do not contain preventive or therapeutic content.

The researchers independently extracted the general characteristics of the included guidelines, year of publication, updating status, method of evidence identification, categories (guidelines, consensus), etc.

The independent evaluators assessed the selected guidelines using the AGREE II instrument. The AGREE II includes 23 key items, grouped into six domains (scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence).⁴ Standardized score of each domain= (obtained score − minimal possible score)/(maximal possible score − minimal possible score) × 100%. Guideline recommendation levels were determined by the distribution of the standardized total scores across the 6 domains. Grade A was recommended, which means all of the standardized score in 6 domains ≥60.00%; Grade B means that the number of domains with the standardized score ≥30.00% was ≥3, and there were domains with the standardized total score <60.00%; Grade C was not recommended, which means the number of domains with a standardized score <30.00% is ≥3.⁵ All reviewers were trained online using the AGREE training tools, discrepancies of >3 scores were discussed with the third appraiser.

The statistical analysis of reliability was performed by SPSS 25.0, and the consistency of the two researchers was tested using the intraclass correlation coefficient (ICC). The coefficient value between 0.75 and 1.00 indicated good consistency.⁶ The mean, median, and range of the standardized scores were calculated separately in each domain. The mean or median ≥50.00% indicated high quality, and the range ≥50.00% indicated large quality difference.

Results

After searching, 254 literatures were preliminarily considered. Nine guidelines were finally included in this study (Figure 1).^7–15

Figure 1 Study selection diagram.

Table 1 summarizes the general characteristics of the guidelines. The study included 8 guidelines and 1 consensus covering 3 continents. Four of the guidelines used grading of recommendations assessment, development and evaluation (GRADE) method.

Table 1 General Characteristics

Table 2 shows the scores for each guideline under the six domains. The overall quality varies greatly. The average score was from 30.56% to 82.72%. The highest scores were in the clarity of presentation domain, while the lowest scores were in the rigor of development domain. The average standardized scores of the six domains were 64.77%, 39.51%, 51.97%, 81.18%, 41.20%, and 30.56% respectively, indicating that the overall qualities of the guidelines included in the domains of “scope and purpose”, “rigor”, and “clarity” were high, while the qualities of “participants”, “application” and “independence” were low. From the perspective of range, except for “clarity”, the ranges of the other five domains were ≥50.00%, indicating that the large quality differences. The 7 guidelines/consensus were Grade B (recommended after modification), the others were Grade C.

Table 2 Domain Scores of the Nine Guidelines Assessed by Using the AGREE-II Instrument (%)

Table 3 shows the intraclass correlation coefficients, 95% CIs, and p values for each domain among the four evaluators. The overall intraclass correlation coefficients ranged from 0.821 to 0.997, indicating good consistency.

Table 3 Inter-Rater Reliability Study Results

CAIM, CHBSA, and ACOG only provided recommendations for tubal pregnancy, SOGC, RCPI, ASRM, and RCOG make clear distinctions of treatment among the various types of ectopic pregnancies (interstitial, abdominal, cervical, ovarian), while MOGGE and NICE did not differentiate. Although the main principles of the recommendations on the diagnosis and management of non-tubal pregnancies are the same, it is difficult to have studies with high level of evidence due to the low incidence. This may be the reason that other guidelines do not provide recommendations for such cases.

The included guidelines reached preliminary consensus on diagnosis, indications and contraindications of methotrexate, expected management, and monitoring.

All guidelines emphasized serum β-human chorionic gonadotropin (β-hCG) and transvaginal scan (TVS) as the main basis for diagnosis. Ectopic pregnancy should be highly suspected when there is still no echogenic cystic structure in the uterine cavity after β-hCG test is positive. When EP is suspected, the uterus and adnexa should be carefully examined.¹⁰ RCOG and NICE clearly suggested that progesterone level was not useful for the diagnosis, whereas the SOGC sets a threshold of 20 nmol/L to predict the viability of pregnancy.

RCOG, ACOG and NICE mention that in some cases “pseudogestational sacs” should be distinguished from intrauterine or extrauterine pregnancies.

RCOG, ACOG, and NICE also mention that, in some cases, there is a fluid collection called “pseudogestational sac”, which should be distinguished from either an intrauterine or extrauterine pregnancy. Suspicion of EP increased significantly in the presence of hemodynamic instability and acute abdominal pain.¹² Regarding drug treatment, all guidelines point out that methotrexate (MTX) is a safe and effective treatment for patients with stable hemodynamics and meeting the follow-up requirements. In terms of surgical treatment, women with stable hemodynamics are most suitable for laparoscopic surgery, and laparotomy should be limited to EP rupture with massive hemorrhage and late abdominal pregnancy with placenta attached to main blood vessels. All guidelines indicate that continuous monitoring is required after treatment until the level of β-hCGturn negative. As for the post-treatment period, RCPI recommends weekly follow-up β-hCG level until negative, and TVS should be rechecked to observe adnexal recovery, while other guidelines recommend measuring β-hCG on days 4 and 7 after administration of MTX. RCOG, RCPI, and ACOG recommend that women should postpone a future pregnancy for at least 3 months after MTX administration. These terms may reflect the differences among different health care policies, which are usually based on local cost–benefit analysis.¹⁶ In addition, CAIM indicates that compared to simple Western medicine or traditional Chinese medicine, the combination of Chinese and Western medicine can reduce the time of β-hCG turning negative, the number of days in hospital, and the time of disappearance of tubal pregnancy. The tubal patency rate is higher, and the possibility of mouth ulcers and gastrointestinal discomfort is also reduced. This guideline provides recommendations for the traditional Chinese medicine (TCM) treatment of tubal pregnancy, which differs from other guidelines.

Discussion

In our study, seven guidelines were comprehensively recommended for Grade B and another two for grade C. According to the criteria of previous studies,¹⁷ the number of fields ≥4 with more than or equal to 50% can be considered as high-quality guidelines. There are four guidelines/consensus that are high-quality.^7,8,10,14

Currently, AGREE II is one of the widely used tools for guideline quality evaluation. Its evaluation dimensions are wide-ranging, and the results can provide a reference for clinical practice and guideline development.

This study comprehensively searched and strictly screened ectopic pregnancy clinical practice guidelines or expert consensus published or updated in recent years. The appraisers screened records and extracted data independently to reduce bias and minimize errors.

This study also has limitations. More evaluators’ opinions may increase the diversity of viewpoints.¹⁸ AGREE II assesses guidelines/consensus from a methodological perspective only and does not address guideline specifics, which may limit the application of the results.

The evaluation results showed that 7 guidelines/consensus were recommended as Grade B, the others were recommended as Grade C. Only the average standardized score of the “clarity” field was ≥50%, and other fields were polarized. In the field of “scope and purpose”, only one guideline¹⁵ had a standardized score <50%, mainly because the general purpose of the guideline was not clearly stated. In the field of “stakeholder involvement”, 7 guidelines^7,9,11–15 had a standardized score of <50.00%. Their main shortcomings were as follows: (1) the specific work of the expert was not described, only the name is provided. This information also included subject categories, division of responsibilities, and so on; (2) The opinions and wishes of the target population (patients, the public, etc) were not collected.

“Rigour of development” is the field that can reflect the quality of the guideline. The standardization scores of the 6 guidelines^9,11–15 in this field were relatively low. The main points were as follows: (1) the retrieval strategy, evidence selection criteria, and evidence evaluation criteria were not described; (2) the method of forming recommendation opinions (such as voting, consensus, Delphi method, etc) was not described; (3) external review before publication; and (4) the method of updating was not explained.

Guidelines in this study all have a high score in the field of “clarity”, indicating that all guidelines meet the criteria for each item in this field.

There is only one guideline with a high score in the field of “applicability”¹⁰ which described the factors that will promote or hinder the application in detail, and provided supporting tools and clear supervision or audit standards.

Only three guidelines^7,14,15 scored ≥50.00% in “independence”field. The main points were the lack of clarification on conflicts of interest and whether funding will affect the content of the guideline.

We have only made a comparison on the methodologies of the guidelines. When evaluating the guidelines in the future, other evaluation tools can be combined to comprehensively evaluate the guidelines from multiple perspectives to improve the practical value of the results.

To sum up, this study carried out methodological quality evaluation on the nine guidelines/consensus based on the AGREE II tool, and found that the overall quality of the ectopic pregnancy guidelines had room for improvement. It is recommended to supplement and improve the four fields of “independence”, “rigor”, “participants” and “application”, especially the “independence” and “application” fields. Only four guidelines^7,8,10,14 had higher scores. In addition to considering more high-quality studies, future research or guideline development may need to take into account unmeasured confounding factors, such as patient preferences, different experiences, levels of ultrasound and laparoscopic surgery and various types of ectopic pregnancies (interstitial, abdominal, cervical, ovarian) to optimize the development process of practice guidelines and thereby improve patient prognosis. Moreover, many studies^19–22 have mentioned the role of mifepristone in the treatment of non-tubal pregnancies especially interstitial pregnancy, such as promoting trophoblast necrosis²⁰ and enhancing the trophoblastic effect of methotrexate,²² which can help doctors make better decisions. Its efficacy in interstitial pregnancies is mainly related to the presence of the endometrium, and the use of mifepristone can interfere with the pregnancy development in this particular site.¹⁹

Conclusions

The overall quality of the ectopic pregnancy guidelines had room for improvement. It is recommended to supplement and improve the four fields of “independence”, “rigor”, “participants” and “application”, especially the “independence” and “application” fields.