Validation of the International Consensus Group Criteria for Slide Review Following Automated Complete Blood Count at Jimma Medical Center, Ethiopia

Introduction

Automations in hematology laboratory changed the discipline dramatically following the introduction of a variety of analyzers that provide a reliable and accurate blood cell count, as a complete blood count (CBC).¹ CBC is a test that measures the number and physical characteristics of cells in the circulation including white blood cells (WBC), red blood cells (RBC), and platelets, to determine the general health status of the patient and screen, diagnose and monitor a variety of diseases and conditions that affect blood cells.²

Generally, there are nine components of CBC named as white WBC count, RBC count, hemoglobin (Hgb), hematocrit (Hct), mean corpuscular volume (MCV), mean corpuscular Hgb (MCH), mean corpuscular Hgb content (MCHC), platelet count, and red cell distribution width (RDW). Depending on the type of analyzer, some of these parameters are directly measured, such as WBC, RBC, Hgb, and MCV, while others are calculated such as Hct.³ With the advancement of automation, quantitative and qualitative information are becoming available in the form of flags, histograms, or scattergrams. In addition to the traditional CBC parameters. Abnormalities in one or more components may indicate the presence of one or more conditions, which often leads to other tests to determine the cause of abnormalities, usually the peripheral blood film (PBF) examination.²

A PBF is a laboratory procedure that involves the cytology of peripheral blood cells smeared on a slide. The diagnostic importance of PBF is enormous and its relevance has not been lessened by advances in hematology automation.⁴ It is an inexpensive but powerful diagnostic tool and in some cases is sufficient to establish a diagnosis alone.⁵

However, some evidence recommended that there is a necessity of reducing PBF examination in light of time and monetary savings,⁶ and others suggested that the CBC and PBF are well correlated and hence the usage of automation is recommended to ease workload and time for patients.⁷

On the other hand, some evidence showed a significant discrepancy between automation and PBF on some parameters and suggested the interpretation of CBC results in light of the PBF examination.^8,9 In addition to its importance in quality assurance to confirm findings of automation, PBF is found to be crucial to solely detect some important morphologic changes.¹⁰

In most resource-limited areas, PBF following CBC is performed without any objective criteria and is mainly triggered by physicians. In 2005, the international consensus group (ICG) led by Dr Berend Houwen suggested criteria for action following automated CBC and WBC differential analysis.¹¹ The group indicated the importance of these criteria for a wide range of hematology laboratories and suggested validation of the rules in individual laboratories before implementation as one size does not fit all due to various reasons like the practice of individual clinics, test volume, the instrument used, and others. The ICG criteria were established based on data from laboratories of developed countries including the US, Canada, Australia, and Europe. Hematological tests are often influenced by different variables including race, age, and gender; dietary habits, exposure to pathogens, ecological factors; condition of assay, variations in instrumentation techniques and laboratory personnel.¹² Thus, it is highly important to validate the criteria in developing countries where infectious diseases are still rampant that can affect blood cell count and morphology. Thus, the current study aimed to validate the consensus group criteria for slide review following automated CBC at JMC, Ethiopia.

Materials and Methods

Study Setting

The study was conducted at Jimma Medical Center Hematology laboratory. The Medical Center is one of the largest health facilities in the country providing service to 20 million people with 1600 staff members and 800 beds. The average daily CBC analysis is about 200. The study was conducted from November 1, 2020 to February 30, 2021 comprising a total of 1685 blood samples from the daily laboratory workload of CBC analysis. To validate the ICG criteria, the minimal sample size suggested by the international society for laboratory hematology (ISLH) was 1000 [11]. An average of 20 samples were randomly selected from the daily workload. The samples were collected in K2-EDTA tubes (Becton Dickinson) and analyzed using Coulter DxH 800 and Sysmex XT-1800 hematology analyzers. Among the total samples, 792 were analyzed by Sysmex XT-1800 while 893 were analyzed by Coulter DxH 800 hematology analyzers.

Study Design

A cross-sectional study design was employed to validate the ICG criteria and to compare the criteria with the existing practice.

Peripheral Blood Film Review

A manual slide review was done on two Wright-stained slides for each sample by two experienced technicians, independently and blinded to the automation results. Positive results were defined based on the ISLH recommendation stated as follows.¹¹ The detailed version of the recommendation with grading system is attached in the Supplementary Data section (Supplementary Data 1).

Morphology

1. RBC morphology at 2+/moderate or greater. The only exception is malaria, where any finding will be considered a positive finding.
2. PLT morphology (giant platelets) at 2+/moderate or greater.
3. Platelet clumps at > rare/occasional.
4. Dohle bodies at 2+/moderate or greater.
5. Toxic granulation at 2+/moderate or greater.
6. Vacuoles at 2+/moderate or greater.

Abnormal Cell Types

1. Blast ≥1%
2. Metamyelocyte >2%
3. Myelocyte/promyelocyte ≥1%
4. Atypical lymphocyte >5%
5. NRBC (Nucleated RBC) ≥1%
6. Plasma cells ≥1

Quality Assurance

The analysis of all samples was according to the standard operating procedure and manufacturer’s instructions. Internal quality control (IQC) of the analyzers was regularly applied every three month and an in-house developed patient sample was used as IQC every morning once per day. Moreover, three times a year, the JMC laboratory participates in an external quality assessment (EQA) run by one world accuracy of Canada, involving complete blood count and peripheral blood film review.

Statistical Analysis

All statistical analyses were performed using SPSS version 20 software. The automated CBC results were sorted into four categories as a truth table based on the correlation between the positive/negative status defined by the ICG rules (Supplementary Data 2) versus the “truth” defined by the slide review. If a rule was triggered, and the smear contained a positive finding, the sample was graded as true positive (TP). If a rule was triggered, but the smear did not contain any positive findings, the sample was graded as false positive (FP). A true negative (TN) grade was given when there was no triggered rule, and the smear did not contain any positive findings. A false negative (FN) grade was given when there was no triggered rule, but the smear contained a positive finding.¹¹

Sensitivity, specificity, accuracy rate, error rate, review rate, positive predictive values, and negative predictive values were calculated based on the following calculation:

Ethical Considerations

Ethical clearance was obtained from Jimma University, Institute of Health Ethical Review Committee (Ref. No. IRB000342/2012). The principles of the Declaration of Helsinki have been followed in conducting the study. Any information concerning the participants was kept confidential and the Authors had no access to information that could identify individual participants during or after data collection. The patient result of the slide review was communicated to the treating physician.

Results

The samples were collected from more than 13 wards including medical, outpatient, emergency, pediatrics, maternity, and oncology. About a fourth of all samples (27%) came from oncology department, whilst samples from ICU and ART clinic were the least frequent. From the total samples, 45% were from males and 55% were from females. The age of the patients ranges from 1 to 90 years.

Analysis of Smear Review Findings

Out of the total 1685 samples, there were 671 (39.8%) positive findings according to the definition of ISLH; 283 (42%) were positive samples from Coulter analyzer, whereas 388 (58%) were positive samples from Sysmex analyzer. The majority of the abnormalities were related to RBC, followed by WBC and platelet abnormalities. Anisocytosis was the most common finding while giant platelet is the least common.

Analysis of the ICG Criteria

The truth table showed that FN and FP results were 2.4% and 4.8%, respectively, for Sysmex analyzer. For Coulter analyzer, the FN and FP rates were 4.6% and 4.7%, respectively. The FN rate was within the acceptable range for both analyzers which is less than 5%. The review rate was also found to be 51% and 31.8% for the two analyzers, respectively. The diagnostic performance of both analyzers based on the ICG criteria and slide review positivity was found to be good with sensitivity and specificity of 95.1% and 90.6% on Sysmex and 85.5% and 93.1% on Coulter analyzer (Table 1).

Table 1 Truth Table Based on the International Consensus Group Criteria

False Negative

The total number of FN occurrences on Sysmex and Coulter analyzer were 24 and 57 and the total number of FN cases were 19 and 41, respectively. The majority of the occurrences on samples from coulter analyzer and Sysmex analyzer were related to RBC abnormalities, particularly, anisocytosis followed by findings of macrocytes. Moreover, white blood cell abnormalities, such as toxic granulation and atypical lymphocyte contributed to the FN rate (Figure 1).

Figure 1 False negative occurrences on Sysmex (A) and Coulter analyzer (B).

False Positive and True Positive

The most frequently triggered rule on Sysmex analyzer was rule number 5 followed by rule no. 7, 9, and 10. Rule no. 7 was the most frequently triggered rule on Coulter analyzer followed by rule no. 9, 10, and 5. Generally, the TP rate was higher on Sysmex analyzer than Coulter analyzer. In total, 18 cases triggered rule no. 5 (WBC count) on Sysmex analyzer, the majority (17/18) of which appears to be because of the neutrophil count (rule no. 17). The same is true on Coulter analyzer where 10 cases triggered rule no. 5 and 19 cases triggered rule no. 17. The second most common triggered rule among the FP cases was rule no. 7 (platelet count) which was 17 and 16 cases on Sysmex and Coulter analyzer, respectively (Table 2).

Table 2 True Positive and False Positive

Comparison Between the Consensus Group Criteria and Physicians’ Triggered Slide Review

From the total samples included in the study, 616 (36.5%) slide reviews were triggered by physicians. The truth table based on physicians’ triggered slide reviews showed different results on both analyzers. The FN rate was unacceptably higher on both analyzers as compared to the ICG criteria (Figure 2). The review rate was also found to be 49% and 25.5% on Sysmex and Coulter analyzers, respectively.

Figure 2 Comparison between the consensus group criteria and physicians’ triggered slide review.

Discussion

A consensus group established by the ISLH suggested indications for slide review and/or manual WBC differential count following automated CBC.¹¹ The group was comprised of 20 experts from multiple developed countries who use multiple instruments. Criteria at participants’ laboratories were discussed and finally combined into 41 rules. These rules have to be tested in different laboratories in the world before use. The present study involved 1685 samples analyzed by two automated instruments at JMC in Ethiopia to test if the recommended criteria are well fitting to laboratories of developing countries.

In this study, evaluation of the ICG review criteria yielded an FN rate of 2.4% and 4.6% for Sysmex and coulter analyzers, respectively. These results were within the acceptable range according to the ICG recommendation of a maximum acceptable FN rate of less than 5%.¹¹ Though anisocytosis is the most common finding that caused FN results, adjustment of rule #15 (RDW rule) may not be necessary as the rate is within the acceptable range. The FP rate on both analyzers was also acceptable when compared to the rate that was found by the ICG, which is 18.6%. Rule #5 and # 17 were the most frequently triggered rules that caused FP results. Increased WBC count (>30 x10⁹/L) rather than decreased WBC (<4 x10⁹/L), particularly the neutrophil count (>20 x10⁹/L) was the frequent reason as expected in the areas where bacterial infection is common.¹³

The College of American Pathologists (CAP) had a Q-probe study of 95,151 CBC determinations performed in 263 institutions and the median PBF review rate was 26.7%, which was suggested as a normative review rate.¹⁴ The present study showed that the review rate for Sysmex and Coulter analyzers were 51% and 31.8%, respectively. The review rate of samples from Sysmex analyzer appears to be higher as compared to the review rate of the Q-probe study by CAP. However, a lower review rate, though desirable, might say nothing about quality. In our case, the TP rate was higher in samples from Sysmex analyzer as compared to the samples analyzed by Coulter. This might be due to the patient mix analyzed by this analyzer, the majority of which are from oncology and emergency department, as compared to the samples analyzed by Coulter machine.

The comparison of our finding with previously published studies showed that the FN rate is higher than the previous findings, in which the rate range from 1.6 to 2.2,^15,16 but lower than studies that reported FN rate ranging from 7.37 to 9.25.^17,19 The FP rate in this study is much better than previous studies, which reported FP rates ranging from 7.2% to 34%.^15,16,18,19 Moreover, the review rate in the present study was found to be comparable with some previous studies, which reported a review rate ranging from 29% to 51%.^15,16,19,20

This study has also compared the ICG criteria and physicians’ triggered slide reviews. The physicians’ triggered slide review practice appears to be arbitrary in the sense that no specific rule is set following automated CBC to indicate slide review. The truth table showed that the FN rate was unacceptably higher among slide reviews triggered by physicians. This indicates that many positive findings are remaining unnoticed due to a lack of objective rules, which in turn might affect patients’ safety.

Conclusion

Generally, the ICG rules set by ISLH are suitable to use in our setting. However, we might still need to modify the rules, particularly to reduce the review rates in light of a shortage of skilled technicians to review large amounts of samples. On the other hand, though the review rate was desirably lower for samples from the Sysmex analyzer, continuing with physicians’ triggered slide review approach might result in increased unnoticed positive findings. It is also necessary to confirm the rules with case mixes proportionally derived from the source population.