Interventions targeting the prescribing and monitoring of vancomycin for hospitalized patients: a systematic review with meta-analysis

Introduction

Vancomycin is an essential antibiotic that has been in use for six decades.¹ Despite sustained use, vancomycin remains an inherently challenging drug to prescribe, due to the need for individualized dosing and requirement for serum-concentration monitoring to ensure efficacy, minimize nephrotoxicity and limit the development of resistant organisms.^2–5 Recommendations on how to dose and monitor vancomycin have evolved over time.⁶ These issues, in addition to the greater public health concern of antimicrobial resistance,⁷ have resulted in a number of professional societies in the US, Japan, and more recently China publishing their own vancomycin guidelines. ^8–10 Significant time and expert engagement goes into the development of these high-caliber guidelines,¹¹ which are sanctioned and advocated by their respective countries.^12,13 These guidelines provide important updated information for clinicians and seek to improve care for patients; however, there is a dearth of information as to how these guidelines should be implemented into practice to fulfill these objectives. The published protocol for the development of clinical practice guidelines for therapeutic drug monitoring (TDM) of vancomycin by the Chinese Pharmacological Society is the only one that provides any advice on implementation.¹⁴ In addition to limited information on implementation strategies of these guidelines, there is scant evidence on which interventions may be best employed and in what combination.

There are a number of published works stating that clinicians in numerous fields of medicine often do not follow guidelines, including prescribing antibiotics for hospitalized patients.^15–18 In an effort to address these problems, strategies have been advocated by peak national bodies concerned with guideline implementation and care improvement, such as the UK National Institute for Health and Care Excellence, the US Institute of Medicine, the Australian National Health and Medical Research Council, and more broadly the Guideline International Network.^12,19–21 Examples of strategies recommended by these bodies include implementation of guidelines, educational meetings, and dissemination of educational material.^22–24 Determining optimal strategies, employed alone or in combination, is critical to inform practice initiatives seeking to translate guidelines and their recommendations into practice. This systematic review aims to evaluate the effect of interventions using education, guideline implementation, and dissemination of educational resources on the dosing and monitoring of vancomycin in hospitalized patients.

Methods

Registration and protocol

The protocol for this systematic review was registered (CRD42016049147) with PROSPERO, (International Prospective Register of Systematic Reviews, Center for Reviews and Dissemination, University of York, UK) in October 2016. A protocol for this review has been published.²⁵ The review has been reported in accordance with the PRISMA (preferred reporting items for systematic reviews and meta-analyses) 2015 statement.^26,27 A PRISMA flow diagram of included studies is presented in Figure 1.

Figure 1 PRISMA study flow diagram.

Research question

Do interventions (alone or in combination) involving education, implementation of guidelines/protocols, or dissemination of educational materials (printed or electronic) improve the prescribing, monitoring, and safety of vancomycin?

Eligibility criteria

Studies included were restricted to the English language. Due to a pilot search suggesting a limited number of randomized controlled trials, no restrictions were placed on study type, which included observational and cohort studies. There were no restrictions on year of publication, with databases searched back to their inception. The studies included required interventions to influence vancomycin prescribing and monitoring, using educational meetings (face to face, online, or continuing education), guideline or protocol implementation, dissemination of educational materials, or multifaceted interventions comprising one or more of these. These interventions were selected as they are commonly recommended implementation strategies that are not cost-prohibitive.^20,28 Excluded studies were those that used pharmacokinetic modeling based on guidelines/protocols/nomograms, compared one guideline directly with another (rather than an intervention to implement the guideline), lacked comparator or baseline data, and where postimplementation outcomes excluded patients not managed in accordance with the new guideline (so as not to bias or misrepresent uptake of the guideline). Studies employing interventions where outcomes were exclusively based on indication and/or duration of vancomycin therapy were also excluded.

Data sources

The database searches were performed in October 2016 using the predefined search strategy and method described in the published protocol of our review.²⁵ The following five databases were searched: Ovid Medline, PubMed, Embase, CINAHL, and the Cochrane Database of Systematic Reviews. In addition, we performed a hand search of reference lists of systematic reviews captured in the original search. We used medical subject headings,²⁹ and their synonyms as search terms. We used syntax suitable to detect different spelling and truncation of search terms for the various databases. Search terms principally related to interventions were “guideline/protocol”, “adherence”, “impact”, “evaluation”, “disseminate”, “implement”, “education”, “lecture”, “tutorial”, “seminar”, “feedback”, “reminder”, “electronic mail”, “smartphone”, “computer”, “personal digital assistant; and outcomes”, “prescribing”, “dosing”, “drug monitoring”, and “monitoring”. This list is not exhaustive: the full search strategy is included as the Supplementary material. The search was rerun in May 2018 to identify any potentially new citations that had been published prior to submission.

Data management and extraction

All citations captured were stored in a dedicated and shared library using EndNote referencing software (version X7.7; Clarivate Analytics, Philadelphia, PA, USA). Titles and abstracts of studies were reviewed and assessed independently by two authors for suitability of inclusion. Two authors (CJP and AJW) independently reviewed the full text of relevant studies, any disagreement was resolved by a third investigator. Studies that satisfied eligibility were included for data extraction. Two authors piloted the data-extraction tool before agreeing on the final tool, which was employed using Excel (Microsoft, Seattle, WA, USA). The data-extraction tool was located in cloud storage (Dropbox version 16.4.30; Dropbox, San Francisco, CA, USA) to enable shared and remote access by authors. Data collected included author, year, country, study design, type of intervention, description of interventions, and outcome measures.

Outcome measures

Data were collected for outcome measures of vancomycin dosing. Loading dosages and maintenance dosages appropriate for renal function were as defined by individual study authors. TDM outcomes were the timing of blood samples at steady-state concentration (ie, blood taken prior to the fourth or fifth dose with 12-hourly dosing in patients with normal renal function),⁹ appropriate timing of trough levels (ie, prior to next dose),⁸ attainment of therapeutic target,^8,10 and frequency of patients with supratherapeutic vancomycin concentration (>20 mg/L, at which likelihood nephrotoxicity increases steeply).³⁰ The safety outcome of frequency of reported nephrotoxicity was also included, defined as an increase in serum creatinine of 0.5 mg/dL or >50% from baseline on two or more consecutive measurements after ≥2 days of vancomycin therapy.³¹

Interventions

We categorized interventions according to the Cochrane Effective Practice and Organization of Care Group (EPCO) taxonomy of health-system interventions. The four categories of this taxonomy are delivery arrangement, financial arrangements, governance arrangement, and implementation strategies. Implementation strategies are further subdivided into interventions targeted at health care workers. In this subdivision, the interventions are audit and feedback, clinical incident monitoring, monitoring the performance and delivery of health care, communities of practice, continuous quality improvement, educational games, educational materials, educational meetings, educational outreach, clinical practice guidelines, interprofessional education, local consensus processes, local opinion leaders, managerial supervision, patient-mediated interventions, public release of performance data, reminders, routine patient-reported outcome measures, and tailored interventions.³² The target cohort of interventions was hospital clinicians. For definition purposes in this review, patients treated by staff who were subject to interventions are referred to as the intervention group. Patients under the care of hospital clinicians that were not subject to interventions are referred to the usual-care group.

Risk of bias

Quality assessment of included studies was performed using ROBINS-I (risk of bias in nonrandomized studies – interventions). ROBINS-I was developed by members of the Cochrane Bias Methods Group and Non-Randomized Studies Methods Group and has been validated.³³ As all studies in this review were nonrandomized and conducted in a health care environment, the ROBINS-I tool was highly suitable. ROBINS-I contains seven domains of bias: due to confounding, selection of participants, classification of interventions, deviations from intended interventions, missing data, measurements of outcomes, and selection of reported results. ROBINS-I provides detailed guidance on categorizing each domain as low risk, moderate risk, serious, or critical risk of bias. ROBINS-I detailed guidance states that the level of risk of bias can only be as good as the highest risk obtained for any one of the seven domains, and it is unlikely that an observational study will be judged less than moderate risk.³³ Two authors (CJP and AJW) independently assessed studies for quality, with any disagreement resolved by a third author (Figure 2).

Figure 2 Quality of included studies: ROBINS-I (risk of bias assessment in nonrandomized studies – interventions).

Statistical analysis

Event rates for intervention and standard care are described using frequencies and proportions and differences described using ORs with 95% CIs in Stata (version 15.1; StataCorp, College Station, TX, USA) using the epitab “cci” command. We performed random-effect meta-analyses for the various study subgroups with inverse-variance weights using the R “meta” package (version 4.9.1) with R software (version 3.4.1; Vienna, Austria). Forest plots were created using RevMan version 5.0 (Nordic Cochrane Center, Copenhagen, Denmark). Heterogeneity was assessed using t² and I². I²=0 represents no heterogeneity, while increasing values represent the presence of heterogeneity. I² values of 25%, 50%, and 75% were defined as low, moderate, and high heterogeneity respectively.³⁴

Results

Search results

The search captured 12,483 records across five databases. Following duplicate removal, 10,036 citations were screened and 93 full-text articles sourced, with 89 subsequently excluded (Figure 1). Four studies met inclusion criteria. This was increased to six after the search was rerun prior to submission. All studies included were observational, and no randomized controlled studies were identified. Studies involving interventions employing single or multifaceted interventions were included.

Quality of studies and risk of bias

Five of six included studies had at least one domain that was assessed as moderate risk of bias, and one study had a serious risk of bias for two domains. No studies had domains ranked as critical risk of bias. Figure 2 shows the assignment of risk of bias for each of the seven domains of each included study. Overall risk of bias for each study is presented in Table 1.

Table 1 Study design and characteristics of included studies Abbreviations: NR, not reported; SVC, serum vancomycin concentrations; TDM, therapeutic drug monitoring; ROBINS-I, risk of bias in nonrandomized studies – interventions.

Characteristics of included studies

Five of the six studies were from the US^{35,36,38–40} and one from Australia.³⁷ Three studies reported the population as number of patients, with 263 in the intervention group and 274 receiving usual care,^35–37 and one study reported treatment courses, with 200 in the intervention group and 279 receiving usual care.³⁸ Two studies that evaluated timing of blood samples for vancomycin assays exclusively reported only the number of concentrations: 387 in the intervention group and 288 receiving usual care.^39,40 Data on characteristics of included studies and details of intervention are summarized in Tables 1 and 2.

Interventions

All interventions involved education meetings.^35–40 Five studies employed multifaceted interventions,^35–38,40 including two or more interventions. Two studies involved implementation of guidance, educational meetings, and dissemination of educational materials.^37,38 Two studies employed guidance and education meetings,³⁶ one utilized education meetings and dissemination of educational material,⁴⁰ and another used educational meetings only.³⁹ Of the four studies using guidance, two employed a clinical practice guideline,^37,38 one a nomogram,³⁶ and one an undefined policy change.³⁵ Dissemination of educational materials was employed in three studies using a pocket reference card (Table 2).^37,38,40 Reported outcomes and effect sizes for studies employing interventions on dosing, monitoring, and nephrotoxicity outcomes are presented in Table 3. Interventions involving implementation of clinical practice guidelines, educational meetings, and dissemination of educational resources had the highest effect on dosing outcomes (effect size 2.76–7.28, P<0.001).^37,38 Furthermore, studies using these three interventions when assessing initial maintenance doses being prescribed appropriate for renal function demonstrated relatively consistent effect sizes: OR 2.76 (95% CI 1.66–4.58, P<0.001)³⁷ and OR 3.36 (95% CI 2.22–5.09, P<0.001).³⁸ Overwhelmingly, the studies employing a composite of implementation of guidelines, educational meetings, and dissemination of educational material also had the greatest effect on TDM outcomes. A notable exception was one study using educational meetings and dissemination of educational material, which produced a greater effect size (OR 4.2, 95% CI 1.16–15.17; P=0.024)⁴⁰ when compared with studies that used three interventions: OR 2.18 (95% CI 1.43–3.32, P<0.001)³⁸ and OR 1.42 (95% CI 0.87–2.32, P=0.162).³⁷

Table 2 Intervention details of included studies Abbreviations: EPOC, Effective Practice and Organisation of Care (Cochrane); NR, not reported; CME, continuing medical education; TDM, therapeutic drug monitoring.

Table 3 Summary of interventions on dosing, monitoring, and safety outcomes Note: *Indicates serum vancomycin concentrations. Abbreviations: NR, not reported; CPG, clinical practice guideline; EM, educational meeting; EPOC, Effective Practice and Organisation of Care (Cochrane).

Outcome measures

Effect of interventions on dosing of vancomycin

The overall effect of interventions on vancomycin dosing was OR 2.50 (95% CI 1.29–4.84, P<0.01). The heterogeneity between studies was high (I2=83%, P<0.01; Figure 3). Three studies measured the impact of interventions on loading doses.^35–37 The overall frequency of receiving a loading dose for patients in the intervention group (112 of 263, 42.6%) compared to those receiving usual care (69 of 274, 25.2%) was not significantly different (OR 2.08, 95% CI 0.49–8.79; P=0.32). High heterogeneity among those studies was present (I2=90%, P<0.01; Figure 3A). There were two studies that measured the effect of interventions on maintenance dosages appropriate for renal function.^37,38 There was a higher frequency of maintenance dosages prescribed for patients in the intervention group (246 of 333, 73.9%) compared to those receiving usual care (183 of 378, 48.4%; OR 3.11, 95% CI 2.26–4.28; P<0.01). There was low heterogeneity between these studies (I2=0, P=0.55; Figure 3B).

Figure 3 Effect of interventions on vancomycin dosing.

Effect of interventions on monitoring of vancomycin

Three studies evaluated the effect of interventions on whether blood samples were collected at steady-state concentration.^37,38,40 There was a higher proportion of concentrations appropriately collected at steady state (196 of 356, 55.1%) for patients in the intervention group compared to those receiving usual care (122 of 314, 38.9%; OR 1.95, 95% CI 1.26–3.02; P<0.01) There was no significant heterogeneity between studies (I2=38%, P=0.20; Figure 4A). Three studies measured the effect of interventions on appropriate timing of trough blood samples for vancomycin assays prior to next dose.^35,38,39 There was no difference between patients in the intervention group (463 of 668, 69.3%) and those receiving usual care (302 of 569, 53.1%; OR 2.02, 95% CI 0.72–5.72; P=0.18), although there was significant heterogeneity between these studies (I2=94%, (P<0.01; Figure 4B).

Figure 4 Effect of interventions on vancomycin therapeutic drug monitoring.

There was no significant difference in patient attainment of therapeutic target between those in the intervention group (161 of 233, 69.1%) and those receiving usual care (144 of 225, 64%; OR 1.50, 95% CI 0.49–4.63; P=0.48). There was also significant heterogeneity between studies (I2=80%, P=0.02; Figure 4C). No association was seen between the frequency of patients attaining potentially toxic supratherapeutic vancomycin levels above target (>20 mg/L) in the intervention group (102 of 233, 43.8%) and those receiving usual care (143 of 225, 63.6%; OR 0.45, 95% CI 0.11–1.83; P=0.26). There was significant heterogeneity between these studies (I2=92%, (P<0.01; Figure 5A).

Figure 5 Effect of interventions on supratherapeutic concentrations and nephrotoxicity in patients receiving vancomycin.

Effect of interventions on frequency of nephrotoxicity

There were two studies reporting the number of patients that experienced nephrotoxicity. No association was observed between patients in the intervention group (23 of 233, 9.9%) and those receiving usual care (29 of 225, 12.9%; OR 0.75, 95% CI 0.42–1.34; P=0.33). There was low heterogeneity between these studies (I2=0, P=0.60; Figure 5B).

Discussion

To our knowledge, this is the first systematic review with meta-analysis to explore the effect of commonly recommended interventions of educational meetings, implementation of guidance, and dissemination of educational materials on vancomycin dosing, monitoring, and nephrotoxicity. We found these interventions combined or used individually had a variable effect on dosing, monitoring, and nephrotoxicity outcomes. All studies employed a constant of educational meetings. A Cochrane review on the effect of educational meetings on professional-practice health care outcomes found that educational meetings had a modest effect (median 6%, IQR 1.8%–15.9%) on these outcomes when compared to no intervention.²² This is broadly consistent with our findings when educational meetings were the sole intervention. While no included study used dissemination of educational material exclusively as an intervention, one study that used this in conjunction with educational meetings demonstrated a much higher effect change of 34%, although this was a small study.⁴⁰ A Cochrane review of the effect of disseminating educational materials to medical officers found a minimally increased effect (median 2%, range 0–11%) when compared to no intervention, but an increased effect (median 13%, range 16%–36%) was observed when interventions were followed up to 9 months.⁴¹

The US Institute of Medicine recommends promoting multifaceted interventions to implement guidelines at individual practitioner and health care system levels.²⁰ However, some authors have expressed strongly that multifaceted interventions are no better when compared to single interventions in changing health care professionals’ behaviour.⁴² This was inconsistent with our findings. Five of the six included studies used multifaceted interventions to improve dosing and monitoring of vancomycin. While the effect of individual interventions when combined do not appear to have had a proportional summative effect, those studies with interventions that specifically employed a guideline, educational meetings, and dissemination of educational materials generally had a much greater composite effect than individual interventional component effects.

Others have stated that providing printed material is a reasonable intervention to consider in any implementation strategy, as the costs are not likely to be prohibitive.⁴³ Based on the findings of this review, we agree with this recommendation for educational material to aid dosing and monitoring of vancomycin. Two studies^37,38 with similar interventions that produced favorable effect size changes also adapted their local vancomycin guidelines from US consensus guidelines. This may be meaningful, as guideline content and usability have also been acknowledged as variables in implementation strategies.²⁰ One of the included studies³⁷ had a very detailed description of its educational component published elsewhere⁴⁴ and stated use of additional interventions, including audit and feedback, local consensus processes, opinion leaders in development of guidelines, and email reminder.^45,46 It is possible these interventions may have augmented some of the generally large effect changes observed within that study.

Interestingly no included studies provided assessment of the local barriers and enablers to effective dosing and monitoring of vancomycin in their institution. Understanding these barriers and enablers can influence the choice of intervention, as has been reported by health care professionals conducting implementation projects in health care, including a project to improve vancomycin dosing and monitoring.⁴⁷ Additionally, no included studies provided any theoretical or behavioral basis for selecting the interventions they employed. Providing a theoretical basis for selecting interventions is increasingly acknowledged as important for any implementation program seeking to influence health-professional behaviour.^48–51 Furthermore, the results of this systematic review and meta-analysis are likely to be applicable to the selection of interventions that optimize the uptake of other health care initiatives in hospitals, particularly those relating to more complex prescribing processes. Another strategy used to implement changes in clinical practice for antibiotic dosing has been the use of clinical decision-support software.⁵² However, a Cochrane review found that while this was useful for the dosing and monitoring of some antibiotics, there was no evidence for vancomycin.⁵³ Implementing a vancomycin nomogram utilizing computerized prescriber-order entry systems has shown to be useful and results in an increased likelihood of prescribers ordering initial regimens that are nomogram adherent.⁵⁴ For institutions operating electronic prescribing, computerized prescriber-order entry is likely to be seen more in the future. Furthermore, smartphone applications provide ready access to contemporary guidance on the use of antibiotics, including vancomycin.⁵⁵ However, data are lacking on whether access to smartphone applications improves dosing and monitoring of vancomycin.

Our study has some limitations. Our search was restricted to English-language citations, so it is possible we did not capture all relevant studies. While we designed a systematic search with the assistance of an experienced medical liaison librarian, the final number of included studies was low, and thus our conclusions are derived from a small number of studies. There was considerable heterogeneity among included studies, in particular for sample size, duration of intervention, details of hospital environment, attitudes, and qualifications and experience of health care professionals. The sustainability of effects once the interventions have concluded is an important question that we were unable to answer in this review. Details about the interventions were at times minimal, limiting utility of comparisons between interventions. Additionally, with the data from this review, we are unable to determine the impact of the various interventions on clinical outcomes, aside from nephrotoxicity. Lastly, in an effort to account for heterogeneity among studies, a random-effect model with weighting using inverse-variance methods was used.⁵⁶

Conclusion

Prolonging the working life of vancomycin is critical in our armamentarium of antibiotics in this era of antimicrobial resistance. Interventions that have favorable effects on dosing and monitoring of vancomycin should be adopted at an individual professional level and more broadly, across health systems, as inappropriate dosing can lead to therapeutic failure, nephrotoxicity, and the emergence of organisms resistant to vancomycin. When designing implementation strategies targeting the dosing and monitoring of vancomycin, multifaceted interventions are more effective. Consideration should also be given to the local barriers and enablers that will have an impact on practice initiatives seeking to improve the use of vancomycin. This review found that multifaceted interventions including guideline implementation, face-to-face educational meetings, and dissemination of educational resources in the form of pocket dosing and TDM cards had a favorable effect on the dosing and monitoring of vancomycin in hospitalized patients.

Acknowledgment

We wish to thank Ms Leila Mohammadi, College and Research Services librarian, Gus Fraenkel Medical Library, Flinders University, Adelaide, Australia, for her assistance and guidance in formulating search strategies for multiple databases.

Author contributions

CJP was responsible for study design, acquisition of data, analysis and interpretation of data, initial drafting, and revision of the manuscript; AJW study design, acquisition of data, analysis and interpretation of data, and critical revision of the manuscript for intellectual content; RAM study design, analysis and interpretation of data, and critical review of the manuscript for intellectual content; RJW study design, analysis and interpretation of data, statistical analysis, and critical review of the manuscript for intellectual content; and DLG study design, acquisition of data, analysis and interpretation of data, and critical review of the manuscript for intellectual content. All authors have read and approved the final version of the manuscript. All authors contributed to data analysis, drafting and revising the article, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Moellering RC. Vancomycin: a 50-year reassessment. Clin Infect Dis. 2006;42(Suppl 1):S3–S4.
2.		Rybak MJ. The pharmacokinetic and pharmacodynamic properties of vancomycin. Clin Infect Dis. 2006;42(Suppl 1):S35–S39.
3.		Giuliano C, Giulano C, Haase KK, Hall R. Use of vancomycin pharmacokinetic-pharmacodynamic properties in the treatment of MRSA infections. Expert Rev Anti Infect Ther. 2010;8(1):95–106.
4.		Ye ZK, Tang HL, Zhai SD. Benefits of therapeutic drug monitoring of vancomycin: a systematic review and meta-analysis. PLoS One. 2013;8(10):e77169.
5.		Howden BP, Davies JK, Johnson PD, Stinear TP, Grayson ML. Reduced vancomycin susceptibility in Staphylococcus aureus, including vancomycin-intermediate and heterogeneous vancomycin-intermediate strains: resistance mechanisms, laboratory detection, and clinical implications. Clin Microbiol Rev. 2010;23(1):99–139.
6.		Avent ML, Vaska VL, Rogers BA, et al. Vancomycin therapeutics and monitoring: a contemporary approach. Intern Med J. 2013;43(2):110–119.
7.		O’Neill J [homepage on the Internet]. Review on Antimicrobial Resistance: Tackling Drug-Resistant Infections Globally. London, England; 2016. Available from https://amr-review.org/. Accessed June 5, 2018.
8.		Rybak M, Lomaestro B, Rotschafer JC, et al. Therapeutic monitoring of vancomycin in adult patients: a consensus review of the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, and the Society of Infectious Diseases Pharmacists. Am J Health Syst Pharm. 2009;66(1):82–98.
9.		Matsumoto K, Takesue Y, Ohmagari N, et al. Practice guidelines for therapeutic drug monitoring of vancomycin: a consensus review of the Japanese Society of Chemotherapy and the Japanese Society of Therapeutic Drug Monitoring. J Infect Chemother. 2013;19(3):365–380.
10.		Ye ZK, Chen YL, Chen K, et al. Therapeutic drug monitoring of vancomycin: a guideline of the Division of Therapeutic Drug Monitoring, Chinese Pharmacological Society. J Antimicrob Chemother. 2016;71(11):3020–3025.
11.		Ye ZK, Li C, Zhai SD. Guidelines for therapeutic drug monitoring of vancomycin: a systematic review. PLoS One. 2014;9(6):e99044.
12.		Qaseem A, Forland F, Macbeth F, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012;156(7):525–531.
13.		Steinbrook R. Improving clinical practice guidelines. JAMA Intern Med. 2014;174(2):181.
14.		Ye ZK, Chen K, Chen YL, Zhai SD. A protocol for developing a clinical practice guideline for therapeutic drug monitoring of vancomycin. J Huazhong Univ Sci Technolog Med Sci. 2016;36(3):469–472.
15.		Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–1465.
16.		Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 1989;321(19):1306–1311.
17.		Chen CL, Lin GA, Bardach NS, et al. Preoperative medical testing in Medicare patients undergoing cataract surgery. N Engl J Med. 2015;372(16):1530–1538.
18.		Mol PG, Rutten WJ, Gans RO, Degener JE, Haaijer-Ruskamp FM. Adherence barriers to antimicrobial treatment guidelines in teaching hospital, the Netherlands. Emerg Infect Dis. 2004;10(3):522–525.
19.		National Institute for Health and Clinical Excellence (NICE) [homepage on the Internet]. PMG20 Developing Nice guidelines: the manual. London, UK: National Institute for Health and Clinical Excellence; 2017. Available from: https://www.nice.org.uk/process/pmg20/resources/developing-nice-guidelines-the-manual-pdf-72286708700869. Accessed October 3, 2018.
20.		Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Clinical practice guidelines we can trust. Washington, DC: National Academies Press. 2011.
21.		National Health and Medical Research Council. Procedures and requirements for meeting the 2011 NHMRC standard for clinical practice guideline; 2011. Melbourne: National Health and Medical Research Council.
22.		Forsetlund L, Bjørndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009 (2):CD003030.
23.		National Health and Medical Research Council (NHMRC). A guide to the development, implemenation and evaluation of clinical practice guidelines. Canberra: National Health and Medical Research Council; 1999.
24.		Giguère A, Légaré F, Grimshaw J, et al. Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;10:CD004398.
25.		Phillips CJ, Wisdom AJ, McKinnon RA, Woodman RJ, Gordon DL. Interventions targeting the prescribing and monitoring of vancomycin for hospitalized patients: A systematic review protocol. Infect Dis Ther. 2017;6(4):557–563.
26.		Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647.
27.		Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.
28.		Scottish Intercollegial Guideline Network. SIGN 50. A Guideline Developers’ Handbook; 2015. Available from: http://www.sign.ac.uk/sign-50.html. Accessed Accessed 5 June 2018.
29.		Kim S, Yeganova L, Wilbur WJ. Meshable: searching PubMed abstracts by utilizing MeSH and MeSH-derived topical terms. Bioinformatics. 2016;32(19):3044–3046.
30.		Bosso JA, Nappi J, Rudisill C, et al. Relationship between vancomycin trough concentrations and nephrotoxicity: a prospective multicenter trial. Antimicrob Agents Chemother. 2011;55(12):5475–5479.
31.		Rybak MJ, Lomaestro BM, Rotschafer JC, et al. Therapeutic monitoring of vancomycin in adults summary of consensus recommendations from the American Society of Health-System Pharmacists, the Infectious Diseases Society of America, and the Society of Infectious Diseases Pharmacists. Pharmacother. 2009;29:1275–1279.
32.		Effective Practice and Organisation of Care (EPOC) [webpage on the Internet]. EPOC Taxonomy; 2015. Available from: https://epoc.cochrane.org/epoc-taxonomy. Accessed 5 June 2018.
33.		Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.
34.		Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003;327(7414):557–560.
35.		Hammond DA, Atkinson LN, James TB, Painter JT, Lusardi K. Effects of staff education and standardizing dosing and collection times on vancomycin trough appropriateness in ward patients. Pharm Pract. 2017;15(2):949.
36.		O’Brien KA, Mok S. Evaluation of the safety of a vancomycin nomogram used to achieve target trough concentrations. Hosp Pharm. 2015;50(10):900–910.
37.		Phillips CJ, Mckinnon RA, Woodman RJ, Gordon DL. Sustained improvement in vancomycin dosing and monitoring post-implementation of guidelines: Results of a three-year follow-up after a multifaceted intervention in an Australian teaching hospital. J Infect Chemother. 2018;24(2):103–109.
38.		Swartling M, Gupta R, Dudas V, Guglielmo BJ. Short term impact of guidelines on vancomycin dosing and therapeutic drug monitoring. Int J Clin Pharm. 2012;34(2):282–285.
39.		Coleman LK, Wilson AS. Impact of nursing education on the proportion of appropriately drawn vancomycin trough concentrations. J Pharm Pract. 2016;29(5):472–474.
40.		Carroll DJ, Austin GE, Stajich GV, Miyahara RK, Murphy JE, Ward ES. Effect of education on the appropriateness of serum drug concentration determination. Ther Drug Monit. 1992;14(1):81–84.
41.		Grudniewicz A, Kealy R, Rodseth RN, Hamid J, Rudoler D, Straus SE. What is the effectiveness of printed educational materials on primary care physician knowledge, behaviour, and patient outcomes: a systematic review and meta-analyses. Implement Sci. 2015;10:164.
42.		Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:152.
43.		Kovacs E, Strobl R, Phillips A, et al. Systematic review and meta-analysis of the effectiveness of implementation strategies for non-communicable disease guidelines in primary health care. J Gen Intern Med. 2018;33(7):1142–1154.
44.		Phillips JC, McKinnon AR, Woodman JR, Gordon LD. Junior doctors’ preparedness to prescribe, monitor, and treat patients with the antibiotic vancomycin in an Australian teaching hospital. J Educ Eval Health Prof. 2017;14:13.
45.		Grimshaw JM, Thomas RE, MacLennan G, et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8(6):1–72.
46.		Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–1230.
47.		Phillips CJ, Marshall AP, Chaves NJ, et al. Experiences of using the Theoretical Domains Framework across diverse clinical environments: a qualitative study. J Multidiscip Healthc. 2015;8:139–146.
48.		Michie S, Johnston M, Abraham C, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.
49.		Michie S, Carey RN, Johnston M, et al. From theory-inspired to theory-based interventions: A protocol for developing and testing a methodology for linking behaviour change techniques to theoretical mechanisms of action. Ann Behav Med. 2016;52:501–512.
50.		Grayson ML, Macesic N, Huang GK, et al. Use of an innovative personality-mindset profiling tool to guide culture-change strategies among different healthcare worker groups. PLoS One. 2015;10(10):e0140509.
51.		Davey P, Peden C, Charani E, Marwick C, Michie S. Time for action-Improving the design and reporting of behaviour change interventions for antimicrobial stewardship in hospitals: Early findings from a systematic review. Int J Antimicrob Agents. 2015;45(3):203–212.
52.		Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson SA. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc. 2010;17(1):25–33.
53.		Gillaizeau F, Chan E, Trinquart L, et al. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2013 (11):CD002894. doi: cd002894.
54.		Mccluggage L, Lee K, Potter T, Dugger R, Pakyz A. Implementation and evaluation of vancomycin nomogram guidelines in a computerized prescriber-order-entry system. Am J Health Syst Pharm. 2010;67(1):70–75.
55.		Burdette SD, Trotman R, Cmar J. Mobile infectious disease references: from the bedside to the beach. Clin Infect Dis. 2012;55(1):114–125.
56.		Riley RD, Higgins JP, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549.

Supplementary material

Search strategy for Ovid Medline

1. Vancomycin/
2. (vancocin or vancomycin).tw,kw.
3. 1 or 2
4. education, continuing/or education, medical, continuing/or education, nursing, continuing/or education, pharmacy, continuing/or education, professional, retraining/
5. Practice Guideline/or Guideline/or Guideline Adherence/
6. guideline*.tw,kw.
7. (guideline* adj3 (adherenc* or evaluat* or introduct* or impact* or effect* or disseminat* or implement* or integrat*)).tw,kw.
8. Electronic Mail/
9. ((writte* or print* or oral or online* or educat*) adj2 (information or material*)).tw,kw.
10. (face to face or face-to-face or train* or lectur* or tutor* or seminar* or workshop* or academic detail*).tw,kw.
11. (opinion leader* or facilitator* or “linking agent*” or champion or “changing agent*”).mp.
12. ((knowlege or research) adj2 (translant* or transfer* or disseminat* or implement* or broker*)).tw,kw.
13. remind*.tw,kw.
14. Feedback/
15. feedback.tw,kw.
16. chart review.tw,kw.
17. Program Evaluation/
18. Quality Improvement/
19. Clinical Protocols/
20. (protocol* or algorithm* or leaflet* or pamphlet*).tw,kw.
21. computers, handheld/or minicomputers/
22. (mobile* or “cell phone*” or “smart phone*” or smartphone*).tw,kw.
23. ((app$1 or application*) adj3 (phone* or mobile* or cell*)).tw,kw.
24. Drug Monitoring/
25. (prescri* or monitor* or dosag* or dosing).ti.
26. or/4-25
27. 3 and 26
28. limit 27 to english language
29. (note or letter or editorial or comment).pt.
30. 28 not 29