Total Clinical Chemistry Laboratory Errors and Evaluation of the Analytical Quality Control Using Sigma Metric for Routine Clinical Chemistry Tests

Introduction

The use of clinical laboratory test results in clinical decisions has become an integral part of clinical medicine and studies showed that laboratory medicine determines 70% of clinical decisions; however, small variations around this figure (60–80%) were reported.^1,2 The quality laboratory service ensures accurate, precise and timely results. The total testing process (TTP) of a laboratory is a complex procedure that includes three phases: the pre-analytical, analytical and the post-analytical. In all phases of TTP, quality indicators (QIs) are used by health laboratories based on the requirements of the International Organization for Standardization (ISO) 15,189:2012.^3,4

Laboratory errors can occur at any stage of the pre-analytical phase to the post-analytical phase of the TTP.⁵ Pre-analytical errors include all errors that occur prior to analysis. Some of the pre-analytical errors include hemolyzed sample, insufficient sample, incorrect label, incorrect requisition, clotted sample and tube broken in centrifuges. The impact of pre-analytical error occurs in the analytical and the post-analytical stage. At the analytical phase where analysis takes place, non-conformity with quality control, calibration failure, random and systematic errors can be occurred.⁶ Common post-analytical errors include failure to report test results, delay in reporting, incorrect calculation, critical results not reported or delayed, and results sent to the wrong patient.⁷

Sigma metric quantifies the analytical performance of a laboratory process as a rate of Defects-Per-Million Opportunities (DPMO). The evaluation of laboratory errors in terms of sigma metric is more meaningful than the number of defects alone. For the analytical process of the laboratory system, the sigma metric analysis identifies errors in quality indicators of the process and provides error corrections based on results. Using the Six Sigma principles, it is possible to assess the quality of laboratory testing processes and the number of quality controls needed to ensure the desired quality.⁸

Attainment of Six Sigma performance represents 3.4 DPMO and the achievement of 3 sigma values is the minimum acceptable quality for a process to be applied.⁹ A higher sigma metric value means fewer analytical errors and fewer acceptable test results are falsely rejected, a lower sigma metric value of the parameters indicates higher analytical errors and many acceptable test results are falsely rejected, which is more difficult to use in the analysis of patient samples. However, low sigma metrics could also mean that there are high true rejections that may occur as a result of inadequate QC data and the short study period.¹⁰

Laboratory errors that occur at any phase of the TTP can directly contribute to increased healthcare costs, decreased patient satisfaction, delayed diagnosis, misdiagnosis and a serious risk to the patient’s health.¹¹ Studies showed that 6.4% to 12% risk of inappropriate care and death occurs due to laboratory errors.¹² Studies in the United States indicated that diagnostic errors occur in 5% of the outpatients and about half of the errors may cause severe harm to patients. This is supported by a study conducted in Malaysia with 3.6% errors.^13,14

Although analytical errors contain fewer errors than pre-and post-analytical phases, analytical quality is still a major issue. For example, studies in Ghana revealed unsatisfactory sigma level (<3) for all analytes.¹⁵ Laboratory personnel should control the whole TTP using QIs and focus on improving extra-laboratory procedures, in particular test selection and interpretation in coordination with other medical staff.¹⁶

The achievement of sustainable laboratory performance in accreditation has also been a major challenge in African countries and there are still gaps in strengthening laboratory services.¹⁷ According to previous studies, the majority of Ethiopian clinical laboratories provide sub-optimal service and the quality of the service is severely compromised, which may lead to the occurrence of several laboratory errors.¹⁸ Despite very few studies on the overall distribution of laboratory errors in Clinical Chemistry, no studies have yet been conducted to evaluate analytical performance using sigma metric in Ethiopia.^19,20 Therefore, the current study will assess the distribution of errors in each phase of the clinical chemistry testing process and the analytical performance of routine chemistry tests using sigma metric.

Materials and Methods

Study Design and Setting

A cross-sectional study was conducted at the Dessie Comprehensive Specialized Hospital Clinical Chemistry Laboratory from 10 February 2020 to 10 June 2020. Dessie Comprehensive Specialized Hospital laboratory was involved in the Stepwise Laboratory Quality Improvement Process towards Accreditation, although its performance was not satisfactory.

Study Variables

The study variables included all phlebotomists, laboratory professionals in the Clinical Chemistry laboratory, Clinical Chemistry test requests with their respective blood samples for routine Clinical Chemistry tests, quarterly external quality control (EQC) and all daily internal quality control (IQC) data from the Clinical Chemistry laboratory during the specified study period. Therefore, the study covered the pre-analytical, analytical and post-analytical phases of the TTP in the Clinical Chemistry Laboratory. However, body fluids other than venous blood samples and not routine clinical chemistry tests were excluded from the sigma metric evaluation.

Data Collection Procedure

A standardized pre-tested assessment checklist was used to evaluate the pre-analytical, analytical and post-analytical phases of the Clinical Chemistry Laboratory. The checklist was prepared using variables from different studies and International Federation of Clinical Chemistry.²¹ The selected data collectors were trained on how to collect all the necessary data in the assessment of the completeness of the standard laboratory request forms (such as, patient age, gender, signature of physicians, location, date and authorized request formats), in the assessment of specimen quality, analytical and post-analytical QIs based on the checklist.

All required data were collected using a checklist in all phases of the TTP. In addition, face-to-face interviews were also used to collect socio-demographic data. Data on pre-analytical variables, specifically laboratory specimens, were collected through observation. Color charts were used to assess the quality of the specimens, which were used to standardize the reports of each data collector.

Furthermore, a routine automatic biochemistry analyzer (DIRU CS-T240, Dirui Industrial Co., Ltd. China) was evaluated by sigma metric and important data generated by the analyzer were extracted by using observation. Daily IQC data collected during the analytical phase were entered in the checklist and record formats. The laboratory was involved in the EQC program by analyzing five different concentration proficiency test samples provided by one world Accuracy and the most recent quarterly EQC laboratory performance report was taken and transferred to the checklist. Post-analytical QI outcomes were collected before the results were delivered to clients, and each variable was entered as a record in the checklist.

Data Quality and Statistical Analysis

The standardized data collection checklist was pre-tested on 30 venous blood specimens with their corresponding test requests for Clinical Chemistry at Boru-Meda Hospital to check its feasibility and ensure the validity of the study tool.²² The checklist was modified based on the pre-test results. The investigator undertook a continuous monitoring and support to ensure complete, consistent, clear and accurate data.

The data were checked for completeness and reliability. After ensuring this, data were entered to EPI info version 7 and transferred to SPSS version 21 (IBM Corporation, Armonk, NY, USA) for analysis. The overall distribution of errors in the Clinical Chemistry Laboratory was summarized by descriptive statistics using frequency tables. The sigma metrics value of each analyte was summarized in tables after extracting Total Allowable Error (%TEa) value of each analyte from Clinical Laboratory Improvement Amendment (CLIA).²³ A chi square test was used to check the presence of statistical difference between the three phases of Clinical Chemistry testing process.

In addition, the chi square test was also used to assess the association between selected independent variables with pre-analytical and post-analytical errors. Bivariate and multivariate logistic regression were used to assess the association between work shift and location of patients with prolonged turnaround time (TAT). The strength of the association was measured using the adjusted odds ratio (AOR) and 95% confidence interval (CI). The p-value <0.05 was considered statistically significant. The chi square test and logistic regression were conducted for selected independent variables (work shift and patient location) because the assumptions were not fulfilled by other independent variables.

Analytical quality control performance for routinely tested 14 analytes [aspartate transaminase (AST), alanine transaminase (ALT), alkaline phosphatase (ALP), urea (UR), creatinine (CRE), glucose (GLU), total cholesterol (TC), triglyceride (TG), high-density lipoprotein (HDL), low-density lipoprotein (LDL), total protein (TP), albumin (ALB), total bilirubin (TBIL) and uric acid (UA)] was evaluated using a sigma metric quality monitoring tool. The TEa source for each of the analytes was CLIA. The bias of all analytes except LDL was calculated using the performance of the Laboratory in the EQC and the peer groups mean. Since LDL was not properly documented in the EQC documentation chart, its bias was calculated using the observed mean from the daily IQC and the target value from the manufacturer. The CV for all analytes was calculated from the observed mean and SD of the daily IQC in the study period. For all analytes, two levels (normal and pathological) of IQC materials were used, and sigma metrics value was calculated separately. The estimates of sigma metrics were optimistic because the quality control materials were not from third parties. The cause of low sigma value (<6) in all parameters was identified using the calculated Quality goal index (QGI). The sigma metrics was calculated with the Sigma = (TEa – Bias%)/CV. In addition, the QGI ratio was calculated using a Bias%/1.5 × CV%.²⁴ A sigma value between 3 and 4 quality requires a multi-rule procedure 1_3S/2_2S/R_4S/4_1S/10x, with four levels of control measurement in two runs (N=4, R=2) or two levels of control measurement in four runs (N=2, R=4). However, <3-sigma cannot be controlled with statistical QC protocols of Westgard rules and the method must be rechecked.²⁵

Results

General Information of the Study

A total of 4719 test requests with venous blood samples were included and 18,636 tests were analyzed in the study. Of the total test requests, 153 (3.2%) were from Emergency department, 879 (18.6%) were from outpatient department (OPD) and 545 (11.6%) were from the impatient department (IPD), but the remaining 3142 (66.6%) test requests were with unknown location. The majority 3423 (72.5%) of the samples were analyzed during the first work shift (morning) from 7 AM to 1 PM and equal number of laboratory professionals were assigned in both shifts. Performance of 14 routine clinical chemistry tests with two levels of IQC material was evaluated using the sigma metric monitoring tool.

Missed Information on Test Requests

Of the total of 4719 clinical chemistry test requests, there were 100% completeness of medical record number (MRN) and test ordered, but the clinical data (100%) of the patients and (77.5%) the signature of physicians was missed in the test request form. Of the total test requests assessed, 16,703 defects (more than 3 defects per request) were found. In addition, of the total number of QIs (42,471) evaluated, 16,703 (39.3%) showed missed information on the laboratory test request (Table 1).

Table 1 Frequency of Missed Information on Laboratory Request Forms

Pre-Analytical Errors Related to Specimen Quality

In the assessment of the quality of the specimens, 111 (2.4%) of the specimens were hemolyzed and 97 (2.1%) of the specimens were icteric. In addition, 45 (1%) of the specimens were incorrectly labelled. Of a total of 51,909 QIs, 314 (0.6%) specimen quality-related errors were observed. Of the total sample-related errors, 35.4% were due to hemolysis and 30.9% were due to excessive icteric sample. For one or more reasons, 223 (4.7%) specimens were not suitable for analysis. Of the total pre-analytical QIs (94,380), 17,017 (18%) pre-analytical errors were identified (Table 2).

Table 2 Analysis of Errors in Pre-Analytical Quality Indicators

Analytical Errors

Of the estimated total 1382 daily IQC, 73 (5.3%) were not performed. Of the 1317 IQC data, 81 (6.2%) were out of the acceptable range. In addition, 15 (23.1%) of the EQC performance was reported as unacceptable. Of the total samples analyzed, 141 (3%) tests were interrupted due to electricity fluctuations and all (100%) non-linearity patient test results were released without retesting. Of the total QIs (13,206) in the analytical phase, 457 (3.5%) analytical errors were identified (Table 3).

Table 3 Analysis of Errors in Analytical Quality Indicators

Post-Analytical Errors

Of the total critical test results, 6 (13.3%) were not immediately reported to physicians. Almost all (99.5%) laboratory test results were not verified and signed. It was also found that 62 (1.3%) and 53 (1.1%) laboratory results were unrecorded and reported out of the established TAT, respectively. Of the total post-analytical QIs (37,797), 4827 (12.8%) post-analytical errors were identified (Table 4).

Table 4 Analysis of Errors in Post-Analytical Quality Indicators

Overall Clinical Chemistry Laboratory Errors

Of the total 145,383 quality indicators, an error rate of 22,301 (15.3%) was found in the three phases of the testing process. The chi square test revealed that there was a statistically significant difference between the error frequency of the three phases (χ2=2153, p <0.0001). The highest frequency of Clinical Chemistry Laboratory errors was 17,017 (76.3%) in the pre-analytical phase, while the second observed error was 4827 (21.6%) in the post-analytical phase, but the lowest frequency 457 (2.1%) was identified in the analytical phase (Table 5).

Table 5 Distribution of Errors Frequency in the Total Testing Process

Location of Sample Collection with Pre- and Post-Analytical Errors

Using the chi square analysis, the location of sample collection was found to be statistically associated with missed information on Clinical Chemistry Laboratory request formats (p<0.0001), hemolysis (p=0.04), icteric (p<0.0001), specimen rejection (p=0.001) and failure to record results (p=0.01) (Table 6).

Table 6 Association of Patient Location with Pre-Analytical and Post-Analytical Errors

Factors Associated with Prolonged TAT

The multivariate logistic regression showed that prolonged TAT was independently associated with the second shift (AOR=8.354, 95% CI (4.453–15.670), p< 0.0001) in which the occurrence of prolonged TAT was 8 times higher in the second shift workers compared to the first shift workers (Table 7).

Table 7 Factors Associated with Prolonged TAT

Sigma Value and Quality Goal Index (QGI) of the Clinical Chemistry Tests

Using CLIA as a source of TEa, the sigma metric values of 14 routine analytes were measured. Based on the measured values, only one analyte (ALP) had above 3 sigma values and the remaining 13 analytes were found to be below 3 sigma values. The lowest sigma value (0.01) was observed in ALB (level-I). In addition, the calculated QGI showed that the presence of imprecision, inaccuracy and both imprecision and inaccuracy. It was found that the majority of the poor performance of analytes in the sigma value was due to the imprecision of the QC (Table 8).

Table 8 Total Allowable Error (TEa), CV, Bias, Sigma Value and QGI of the Routine Clinical Chemistry Tests

Quality Control Strategy Based on Sigma Values

Of all the analytes, only ALP had a sigma value of >3. The Westgard rule was established for ALP only and the remaining analytes showed poor performance in which the methods should be rechecked prior to the analysis of patient samples. The Westgard multi rule selected for ALP was 1_3S/2_2S/R_4S/4_1S/10x. Although a multi rule was applied for ALP, the QGI indicated an imprecision that requires a close monitoring and troubleshooting in the daily IQC (Table 9).

Table 9 Quality Control Strategy Based on the Sigma Value of the Test Parameters

Discussion

Although most modern medical laboratories have joined automation, it remains a challenge to ensure accurate and precise laboratory results.²⁶ Therefore, the current study used relatively comprehensive QIs to assess the total errors in the Clinical Chemistry Laboratory.

In the current study, the majority of the pre-analytical errors found were incompleteness of the required information on the test request form (39.3%). This finding was consistent with the study conducted in Hawassa, Ethiopia²⁷ (39.4%), but lower than a study conducted in Gondar, Ethiopia²⁰ with an overall incompleteness of 49.9%. In contrast, the current finding was much higher than studies in Nigeria,²⁸ Kenya²⁹ and Uganda³⁰ with an incompleteness rate of 14.3%, 22.7% and 17.9%, respectively. This disparity may be due to variation in quality indicators, study designs, study periods, awareness and experience of clinicians on the value of patient information.

Of the total number of QIs, 39.3% showed missed information on Clinical Chemistry test requests. This finding was higher than the study conducted in Ethiopia³¹ and Nigeria²⁸ with an overall incompleteness of 8.7% and 10.5%, respectively. This large variation may be due to differences in sample size, operational definition, test requests ordered by inexperienced staffs and lack of commitment to complete the required information.

Based on our study, the clinical history of the patient was not stated in all 4719 (100%) requests which were consistent with the study conducted in Gondar, Ethiopia with 99% incompleteness,²⁰ but higher than studies conducted in Addis Ababa, Ethiopia (72.6%).¹⁹ In addition, signature of physicians (77.5%) and location of patients (66.2%) information were also incomplete in the test request. These findings showed a wide difference from previous studies conducted in Gondar (38.7% and 1.8%),²⁰ Addis Ababa (30.4% and 1.1%)¹⁹ and Nigeria (19.8% and 20.1%)³² respectively. The higher incomplete information in the current study may be due to lack of commitment, failure to create awareness and workload.

In this study, hemolysis was the major cause of sample rejection (35.4%) which was inconsistent with studies conducted in Ghana (17.6%).³³ The increased percentage of hemolysis may be due to untrained phlebotomists, high workload and lack of regular monitoring and support.

According to this study, the prevalence of sample rejection was 4.7%, which was consistent with studies in Egypt³⁴ with a rejection rate of 4.6%. However, it was higher than studies conducted in India (3.45%),³⁵ South Africa (1.46%),³⁶ Turkey (0.65%),³⁷ Saudi Arabia (2.07%)³⁸ and Ethiopia (1.4%).³⁹ The higher rate of rejection may be due to untrained phlebotomists, lack of cooperation and communication, incompetence, improper processing of specimens.

In the current study, the total pre-analytical error (18%) was lower than studies conducted in Iraq (39%)⁴⁰ and Egypt (43.7%),⁴¹ but it was much higher than studies in Ghana (3.7%),³³ Saudi Arabia (3.15%),⁴² Tunisia (7.7%),⁴³ Greece (1.94%)⁴⁴ and India (0.15%).⁴⁵ This may be due to differences in the QIs, sample size, laboratory facilities and the experience of health professionals.

Our study found a lower frequency of 6.2% IQC failure compared to the study conducted in Pakistan (32%),⁴⁶ but higher than the study conducted in Ethiopia (2.95%)¹⁹ and India (0.6%).⁴⁷ The difference between findings may be due to variation in equipment and laboratory personnel performance, storage condition of QC materials, the reconstitution of lyophilized QC materials and availability of in-house mean/SD.

The current study found a 23.1% failure of the proficiency testing, which was inconsistent with a study conducted in Pakistan (5.4%).⁴⁸ This higher level of non-conformity with EQC proficiency testing may be due to improper transport/storage of EQC samples, improper reconstitution of the samples, analyzing by faulty instruments, untrained and incompetent laboratory staffs, reporting with unacceptable unit.

In the current study, 99.5% of the test results were not verified and signed by independent reviewers. This finding was far higher than the study conducted in Kenya (13.1%).⁴⁹ This may be related to non-committed laboratory staffs, the workload of the laboratory and failure to adhere to the laboratory quality policy. In addition, 1.1% of laboratory results were reported out of the established TAT, which was higher than the frequency of delayed TAT in Pakistan (0.003%).⁴⁸ The prolonged TAT in this study may be due to the unavailability of Laboratory Information System (LIS), electric interruption, equipment malfunction and stock out of distilled waters.

The current study found that the prevalence of errors in the TTP was 15.3%. This finding was lower than the study conducted in Addis Ababa, Ethiopia (33.1%)¹⁹ but higher than the study conducted in Saudi Arabia (4.35%)⁴² and Ghana (4.7%).³³ This may be due to difference in the infrastructure of the laboratory, regular monitoring and evaluation, availability of functional LIS, competency of laboratory staffs, in-house mean/SD, variation in the study designs, operationalization of variables and QIs.

Although IQC ensures a continuous monitoring of the analytical method, the exact number of errors that occurred during the analytical phase cannot be evaluated.⁵⁰ As a result, an effective quantifying sigma metrics tool was selected and the sigma metrics value for each test parameter showed that analytical quality is still a major issue.

In the current study, the sigma metric value of most analytes was unsatisfactory. ALP (Level-I (3.9) and II (3.8) IQC) was the only analyte with a sigma value above 3. The remaining 13 analytes (AST, ALT, UR, CRE, GLU, TG, TC, HDL, LDL, TP, ALB, TBIL and UA) had low sigma values <3. This finding was inconsistent with a study conducted in China⁵¹ that reported >3 sigma values for all test parameters (AST, ALT, ALP, UR, CRE, GLU, TC, TG, HDL, LDL, TBIL, UA, TP and ALB). This may be due to differences in the performance of analyzer, poorly stored reagents and QC materials, study period, failure to perform regular maintenance of analyzers and competency of laboratory staffs. In addition, the calculated QGI showed that the presence of imprecision, inaccuracy and both imprecision and inaccuracy. This may be due to power fluctuation, incorrect calibration, improper reconstitution of QC/calibration materials and interferences.

The current study found <2 sigma value for 11 test parameters at two IQC levels for (UR, CRE, GLU, TG, TC, HDL, LDL, TP, ALB, TBIL and UA). This result was consistent with a study conducted in Ghana¹⁵ for (GLU, TG, TC, HDL, UR, CRE, TP and AST) and in Sudan⁵² for UR and CRE. It was also comparable to the study conducted in India⁵³ for ALT (Level-I), TBIL (Level-I), CRE (Level-I) and UR (Level-I and II). In contrast, higher sigma values >3 were reported in China⁵¹ and Turkey⁵⁴ for ALB, GLU, UA, TC, TG and CRE. This disparity in the sigma metrics value may be due to differences in analytical methods, different IQC materials, different proficiency testing bodies and manufacturers.

Conclusion

The study found a high frequency of errors in the Clinical Chemistry Laboratory and most of the errors were observed in the pre-analytical phase. The majority of pre-analytical errors were significantly associated with the location of the sample collection. In addition, with the exception of the ALP, the sigma metric values for all routine clinical chemistry tests were below the standard (<3). Although the sigma metric value of ALP was within an acceptable range, a multi-Westgard rule (13S/2_2S/R_4S/4_1S/10x) should be implemented under close monitoring rather than a single rule.

Specimen collectors, particularly in the IPD should be trained in the laboratory quality management system. Besides the existing manual recording system, the laboratory should provide an electronic backup recording system. In addition, the sigma metric should be included as one of the quality indicators of the Clinical Chemistry Laboratory and the DMAIC Process (Define, Measure, Analyze, Improve and Control) for the improvement of the laboratory services. A regular monitoring and evaluation should be carried out in compliance with 15,189 ISO standards of the clinical laboratory. In addition, the clinicians should be encouraged to use the reference change value to help overcome all the noise generated by poor instrument performance.