The Impact of Information Quality of Antimicrobial Susceptibility Test Report on the Rational Antimicrobial Use: A Retrospective Study

Introduction

Antimicrobial resistance (AMR) occurs when bacteria, viruses, fungi and parasites no longer respond to medicine, making infections harder to treat and increasing the risk of disease, severe illness and death.¹ Considering that the abuse of antimicrobials is a major cause of AMR, comprehensive measures have been utilized for decades to ensure rational antimicrobial use.

Factors driving antimicrobial use include clinical diagnosis, infection severity, pathogens, antimicrobial susceptibility test (AST) results, healthcare provider knowledge and attitude.^2,3 AST-guided antimicrobial use, as part of antimicrobial stewardship, predicts the clinical response of pathogenic bacteria and allows for modification to adequate antimicrobials, improving treatment outcomes, reducing collateral harm, and avoiding unnecessary use of newer drugs.^4–7 The Chinese guideline “Guiding Principles of Clinical Application of Antibacterial Drugs (2015 version)” regulates that pathogenic examination rates before prescribing non-restricted, restricted and special antimicrobials should be more than 80%, 50% and 30%, respectively, emphasizing the importance of AST in antimicrobial use.⁸

However, the guiding significance of AST is sometimes limited.⁹ One reason is that physicians suffer from information overload, which means that physicians do not have enough energy to absorb all information provided by the AST report because of an excessive amount of terminal clinical data.^9,10 Another is that AST report does not have the same absolute validity as most biochemical or haematological reports.^9,11 For example, the significance and treatment options of 10 mmol/L fasting blood glucose level are clear, indicating the risk of developing diabetes, whereas the guiding significance of AST report is absolute only when the isolate is the cause of infection.⁹

According to the previous studies, it was hypothesized that improving information quality (IQ) of AST report was able to reduce information overload and increase its guiding significance, which in turn promoted the rational antimicrobial use. Firstly, higher IQ facilitated efficient information processing, while low-quality information, characterized by chaotic structure, inaccuracies and irrelevance, increased user burden and led to information overload.^12–14 Thus, it could be speculated that improving IQ of AST report provides more useful information for physicians and alleviates information overload. Secondly, previous researches revealed that high-quality information served as a valuable reference for action.¹² By enhancing the IQ of AST report, physicians could receive more valuable and comprehendible information, making AST a better guide for prescribing antimicrobials.^9,15

Accordingly, the impact of IQ of AST report on the rational antimicrobial use is worth studying. Peter Drucker’s idea of “no measurement, no management” emphasized the importance of measuring IQ of AST reports as a first step towards improvement. While previous research has identified IQ issues in AST report (such as over-reporting, mistaken reporting, and insufficient interpretation), there is no evidence on the comprehensive measurement of IQ in these reports.^9,16 Therefore, this study firstly comprehensively measured the IQ of AST report to identify the key points for improvement. Furthermore, there was no evidence on the impact of IQ of AST report on the rational antimicrobial use. Thus, this study secondly explored whether higher IQ of AST reports promoted rational antimicrobial use, aiming to provide evidence for the necessity to improve IQ of AST reports, improve its clinical applicability and promote the rational antimicrobial use to fight AMR.

The Researches of Definition and Measurement of IQ

The definition of IQ focused on accuracy originally and was expanded to a multi-dimensional concept in related research.¹⁷ Diane S. proposed two concepts of IQ: conformity to specification and meeting or exceeding consumer expectations.¹⁸ These concepts led to two approaches for identifying and measuring IQ. The first approach was based on specifications such as policy documents, industry standards, and guidelines.^19–21 The second approach was based on consumer expectations and involved indicators formed by the needs, satisfaction, and experiences of information users and stakeholders.^21,22 Numerous researches have developed frameworks to measure IQ with various dimensions.^18,19,23 However, no general agreement existed either on which set of dimensions defined IQ or on the exact meaning of each dimension.¹⁹ Batini C. et al have reviewed important researches and proposed a basic set of IQ dimensions, including accuracy, completeness, consistency and timeliness.¹⁹ Other dimensions frequently mentioned in literatures included conciseness, readability, accessibility, usefulness, reliability, security and interactivity.²⁴

In the medical field, high-quality information is critical for quality health care and for effective and efficient management of the health care system,¹⁵ and a common way of identifying and measuring IQ is based on the specification.¹⁵ To evaluate IQ in laboratory medicine, the International Federation of Clinical Chemistry and Laboratory Medicine has developed a formal and proactive system based on evidence of adherence to specifications.²⁵ Many countries have established nationwide health data center to drive improvements in the quality of health information, such as National Health Service Digital in England, Canadian Institute for Health Information in Canada, and the Health Information and Quality Authority in Ireland.²¹ One of the functions of these institutions is setting standards and assessing adherence to standards for quality improvement.²¹ The dimensions that were frequently mentioned in the medical and medical informatics literature included accuracy, currency, completeness, readability, usefulness, confidentiality, representational, interpretability and timeliness.¹⁸

Methods

Study Design

Data Collection

This retrospective study was carried out in a tertiary hospital in Jiangxi Province, southeast of China. Participants of the study were inpatients with diagnosis of all infections. The AST report, antimicrobials prescribed after reporting AST results and patient characteristics of participants were collected. Firstly, the AST reports were collected under the following conditions: 1. The report time was between 1 January 2019 and 30 June 2021. 2. The bacteria for detection were extracted from a sterile-site specimen, which was more likely to be the causative agent of infection, and mentioned in Clinical and Laboratory Standards Institute (CLSI) M100. 3. It was the first AST report during hospitalization. Secondly, antimicrobials prescribed after reporting AST results were collected under the following conditions: 1. If there was only one AST report during hospitalization, the antimicrobials prescribed after reporting AST results were collected. 2. If there were two or more AST reports and the time interval between the first and second reports was seven days or more, the antimicrobials prescribed between the first and second reports were collected. 3. If the time interval between the first and second reports was less than seven days, the antimicrobials prescribed within seven days after the first report were collected. All the information was exported from hospital’s electronic information system. Ultimately, a total of 709 cases were included in the study.

AST Methods

Species identification and AST of all isolates were conducted using the Vitek 2 system (bioMérieux, Marcy I’Etoile, France) at the hospital’s clinical laboratory department. The protocols for identification and susceptibility testing of isolates in China were the same as those followed by the Central Laboratory of International Health Management Association.²⁶ AST was performed by disk diffusion method and broth microdilution method in accordance with the CLSI recommendations.

Variables and Measurement

IQ of AST

Firstly, IQ indicators were established based on relevant literature, including a total IQ and four sub IQ: completeness, usefulness, accuracy and consistency^{15,18,19,22,27} (All indicators were presented in Appendix 1). The criteria for evaluating these indicators were derived from specifications, including CLSI M100 and the related standardization of AST report in China, which incorporated requirements from CLSI M100 and China’s specific situation.^28–31 The basis for inclusion, definition and calculation formulas for these indicators were as follows:

The total IQ. To evaluate the overall situation, the total IQ was defined as the degree to which the information of AST report meeting the requirements of all criteria in the four sub IQ. Simple ratio was used as the function forms to assess the level of the total IQ and four sub IQ according to Batini C’s research.²⁰ The calculation formula of the total IQ can be found in Table 1.

Table 1 The Calculation Formulas of IQ Indicators and the Adherence Rate

Completeness. Completeness was chosen because it was one of the basic dimensions according to the research of Batini et al¹⁹ and incompleteness was a prominent problem in AST report such as incomplete annotation information.^15,19,32 Completeness was defined as the degree to which the information that guidelines recommended AST reporting was given in AST report, with reference to Batini et al’s definition, that was the degree to which a given data collection included data describing the corresponding set of real-world objects.¹⁹ In this study, “information that guidelines recommended AST reporting” was considered as the data describing the corresponding set of real-world objects. The completeness of the AST report depended on whether this information was provided. The calculation formula of completeness can be seen in Table 1.

Usefulness. Usefulness was chosen because it referred to the ability to reduce information overload and improve understanding, enabling users to make informed choices,^13,14,33 and research has pointed out that AST report existed irrelevant information such as reporting nitrofurantoin for bloodstream infection.¹⁶ In this study, usefulness was defined as the degree to which the information given in the AST report was in the range of the information recommended by guidelines, because guidelines have regulated the useful information in AST report.^28,34 For example, antimicrobials listed in group A of CLSI M100 were considered useful, while daptomycin included in an AST report for a respiratory specimen was deemed useless.³¹ The calculation formula of usefulness can be seen in Table 1.

Accuracy. Accuracy was chosen because it was one of the basic dimensions according to the research of Batini et al, and inaccuracy was a prominent problem in AST report such as the errors of interpretive categories (ie sensitive, intermediary or resistance) and bacterial identification.^16,19 Accuracy, in this context, was defined as the degree to which the information in AST report was correct according to the guidelines, with reference to Wang’s definition, that was the extent to which data were correct, reliable and certified.²² For example, guidelines gave the standards of antimicrobial’s generic name, susceptibility testing methods and breakpoints. The information in AST report was accurate information if the same as the standard information in guidelines. The calculation formula of accuracy can be found in Table 1.

Consistency. Consistency was chosen because it was one of the basic dimensions according to the research of Batini et al¹⁹ and CLSI M100 recommended to assess the consistency of information in AST report.^19,31 Consistency was defined as the internal consistency between the information of AST report with reference to the definition of Batini et al¹⁹ such as the consistency of the results from individual agents within a specific drug class and the established hierarchy of activity rules. The calculation formula of consistency is presented in Table 1.

Antimicrobial Adherence

Secondly, antimicrobial adherence was used to measure the rational antimicrobial use based on the interpretive categories of AST report. Medication adherence was generally defined as the extent to which the patient’s actual history of drug administration corresponds to the prescribed regimen.³⁵ In this study, it was the evaluation of antimicrobial prescription behavior, thus antimicrobial adherence was defined as the antimicrobial prescribed was effective against isolated bacteria with reference to the interpretive categories of AST report, which included direct and indirect situations. Direct adherence was defined as using antimicrobials to which the bacterium was susceptible or using antimicrobials to which the bacterium was intermediate without reporting susceptible results. Indirect adherence was defined as using antimicrobial that was not reported but had a susceptible result that could be inferred from equivalent agents in CLSI M100 or using antimicrobial with an inferred intermediate result according to equivalent agents in CLSI M100 without reporting susceptible and inferred susceptible results (The summary of equivalent agents in CLSI M100 was presented in Appendix 2).³¹ The adherence rate was used to measure the level of antimicrobial adherence. The calculation formula is in Table 1.

Data Analysis

The analysis of the impact of IQ of AST report on the rational antimicrobial use used the total IQ and four sub IQ as independent variables, and the adherence rate as the dependent variable. A fractional logit regression model (FLRM) was chosen to evaluate the relationship, as it is commonly used for dependent variables defined on the unit interval of 0 ≤ y ≤ 1.³⁶ The functional form for the conditional mean of the fractional outcome was , where the nonlinear function G() ensured that predictions lie inside the unit interval. The RESET test, which was proposed by Papke and Wooldridge (1996), was used to detect whether the model had specification problems, and P-values >0.10 indicated that the model was appropriate specified at 10%.³⁷ In nonlinear models, the magnitude of the change in the dependent variable caused by a unit change in the predictor varied with the starting level of the latter. The procedure for the estimation of average partial effects (APE) within the FLRM was particularly appealing due to the fact that its identification required no assumptions in terms of serial dependence in the dependent variable of the economic magnitude of the relations of interest.³⁸ The interpretation of APE was similar to that of linear regression coefficients.³⁹

We added some control variables to exclude other possible impact factors of antimicrobials use. Age and sex were used to control the impact of patient characteristics. The Age-adjusted Charlson Comorbidity Indicator (ACCI) was used to control the impact of disease severity.⁴⁰ Average Administration Time (AAT) and Total Numbers of Antimicrobials (TNA) were used to control the impact of length and number of prescriptions, respectively.⁴¹

Two FLRM equations were built to evaluate the impact of the total IQ and four sub IQ separately, which were as follows:

(1)

The computation of the IQ of AST report and the antimicrobial adherence were implemented in Python programming language using PyCharm 2022 community edition. Equations (1) and (2) were modeled using the “fractional logit regression” package in R (The Python code was presented in Appendix 3). The dataset of FLRM was attached in Appendix 4.

Ethics Statement

The study was approved by the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology (2021-S063). As the study used anonymous, pooled, and retrospective data and all the collection of patient information was used for matching the information from different databases, the Ethics Committee waived the need for participants to provide written informed consent. The study complied with the Declaration of Helsinki.

Results

The characteristics of variables are described in Table 2. The mean age was 53.32 years, 57.55% were male (N = 709). The mean ACCI score was 4.28 ± 2.85. About 88.31% had at least one comorbidity. Averagely, 4 prescriptions were used after reporting AST results, and the mean duration of every prescription was 3.50 days.

Table 2 Descriptive Statistics of Patient Characteristics, IQ Indicators and Adherence Rate

The median of the total IQ was 0.7345. Among the dimensions, completeness had the worst performance with the lowest median (0.6082), followed by accuracy (median = 0.8966), usefulness (median = 0.9167), and consistency (median = 1.0000). The median adherence rate was 0.5714, which indicated that overall 57.14% prescriptions were adherence to AST reports.

The results of FLRM about the impact of the total IQ and four sub IQ on the adherence rate are illustrated in Table 3, respectively. With a P-value above 0.05, the RESET test indicated that the functional form used by the FLRM of Equations (1) and (2) was appropriately specified.³⁹ The coefficients showed that the total IQ had no statistically significant impact on the adherence rate with P-value >0.05. For the dimensions, the positive relationship between usefulness, accuracy and the adherence rate were all statistically significant at 1% level. When usefulness increased by 0.1 units, the adherence rate increased by 0.100. Accuracy and consistency also had similar effects, increasing the adherence rate by 0.094 and 0.012, respectively. Completeness had no statistically significant impact on the adherence rate with P-value >0.05.

Table 3 The Results of the Impact of the IQ of AST Report on the Adherence Rate

Discussion

This study was the first to comprehensively evaluate the IQ of AST report and explore the impact of the IQ of AST on rational antimicrobial use. The results showed that the total IQ of AST report had no statistically significant impact on the antimicrobial adherence, which indicated that the current level of IQ played a limited role in the reduction of information overload and improvements in guiding significance for rational antimicrobial use. Therefore, it was necessary to identify specific problems in the analysis of the results of four dimensions.

Usefulness had a significant positive impact on the antimicrobial adherence. This was because that the increase of useful information was able to decrease information overload, which in turn prompted physicians to place more emphasis on AST report.¹³ Hence, it was necessary to increase information recommended by guidelines such as equivalent agents in CLSI M100 and the pharmacotherapy recommendation of multidrug resistant bacteria.^31,42

Accuracy had a significant positive impact on antimicrobial adherence. Linda W. Byrd found that the accuracy attribute of IQ was positively and significantly related to the safety, collaboration and professionalism attribute of healthcare quality.³² In this study, accurately reporting break points and interpretive categories was able to increase physician trust to AST report and improve the guidance for prescribing antimicrobials.¹⁵

Consistency had a significant positive impact on antimicrobial adherence. Gail Keenan et al found that improving the consistency of core information in electronic health records had the potential to increase the validity and reliability of health information.⁴³ In this study, the increase of consistency, which meant that the antimicrobials given in AST report were related to the detected bacteria and the interpretive categories, was consistent with the established hierarchy of activity rules, also increased the validity of the information.

Completeness had no statistically significant impact on the antimicrobial adherence, which indicated that reporting more information recommended by guidelines had no impact on the antimicrobial adherence at the current level. One reason for this was that the information in the AST reports was standardized and lacked instructional details. The completeness of the AST reports was relatively consistent with 0.1628 interquartile range, indicating that the information was homogenized. A survey of laboratory users found that 70% general practitioners and 72% hospital doctors requested more interpretation of AST results.⁴⁴ Nevertheless, less than 50% AST reports gave the interpretation comments in this study, and most of them were homogeneous annotations of the full name of S, I, R, intrinsically resistant and the specific name of multidrug resistant bacteria. In the 50 of 709 samples, physicians prescribed antimicrobials to which the bacterium was intrinsically resistant after reporting AST results, 28 (56.00%) of which have given the annotation of intrinsic resistance in the AST reports, whereas physicians still prescribed these ineffective antimicrobials. This phenomenon revealed that physicians did not put more value on the annotations, likely due to the standardized and uninformative nature of the information provided.

This study found that the median of the total IQ was 0.7345, which had scope to improve. Poor performance was observed in completeness and accuracy, consistent with previous studies highlighting issues of incompleteness and inaccuracy in healthcare information.^16,45,46 A case study found that 4.8% case record forms of patient data were inaccuracy and 3.3% was incomplete.⁴⁶ From the above analysis, it was important to improve usefulness, accuracy and consistency of information of AST reports according to the relative guidelines. For completeness, the improvements in the homogenization of information and the increase in the instructive and specific interpretation such as treatment recommendation were necessary.^9,11,47 The measurements of quality improvements were assessing IQ of AST on a regular basis and identifying and acting on problem.⁴⁸ In addition, selectively reporting antimicrobials with reference to the relative guidelines was necessary to improve the guiding significance of AST report.⁴⁹ The above measurements were in hopes of AST report playing a greater role in promoting the rational antimicrobial use and mitigating AMR.

The limitations of this study were as follows. Firstly, some information such as bacteriostatic zone diameter and MIC value in AST results did not have a reference standard, which was not included in the accuracy analysis, leading to the incompleteness of the analysis of accuracy. Secondly, we only evaluated the appropriateness of antimicrobial selection. Dose and dosage form were not included in the measurement of rational antimicrobial use considering that AST report did not give the corresponding information as a reference, while they were also parts of the rational use of antibiotics and could be considered as another research point. Thirdly, physicians’ prescribing habits and antimicrobial stewardship policy were not included as control variables in the evaluation of the impact of IQ of AST report on the rational antimicrobial use because the data were not available, which were also important factors of antimicrobial use.

Conclusion

The study confirmed that usefulness, accuracy and consistency had significant positive impacts on the rational antimicrobial use, although the total IQ and completeness had no statistically significant impact on the rational antimicrobial use. Thus, it was necessary to improve the usefulness, accuracy and consistency of information of AST report. For completeness, reducing homogenization and increasing instructive information were necessary. The above improvements were able to enhance the clinical applicability of AST reports, thus promoting the rational antimicrobial use to reduce AMR.