Back to Journals » Advances in Medical Education and Practice » Volume 5

Clinical performance feedback and quality improvement opportunities for perioperative physicians

Authors Kaye A, Okanlawon O, Urman R

Received 10 February 2014

Accepted for publication 11 March 2014

Published 3 May 2014 Volume 2014:5 Pages 115—123

DOI https://doi.org/10.2147/AMEP.S62165

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3



Alan David Kaye,1 Olutoyin J Okanlawon,2 Richard D Urman2

1Department of Anesthesiology, Louisiana State University School of Medicine, New Orleans, LA, 2Department of Anesthesiology, Perioperative and Pain Medicine, Brigham and Women's Hospital, Boston MA, USA

Abstract: Clinical performance feedback is an important component of the ongoing development and education of health care practitioners. For physicians, feedback about their clinical practice and outcomes is central to developing both confidence and competence at all stages of their medical careers. Cultural and financial infrastructures need to be in place, and the concept of feedback needs to be readily embraced and encouraged by clinical leadership and other stakeholders. The "buy-in" includes the expectation and view that feedback occurs on a routine basis, and those engaged in the process are both encouraged to participate and held accountable. Feedback must be part of an overarching quality improvement and physician education agenda; it is not meant to be an isolated, fragmented initiative that is typically undermined by lack of resources or systemic barriers to gaining improvement within programs. Effective feedback should be an integral part of clinical practice. Anesthesiologists and other perioperative physicians are identifying specialty-specific indicators that can be used when creating a broader quality improvement agenda. Placing a more immediate formal feedback strategy that focuses on goal-oriented behavior is rapidly becoming a mainstay. Physicians may use their individual feedback reports for reflection and designing personal development plans as lifelong learners and leaders in improving patient care.

Keywords: physician education, outcomes measurement, performance improvement, anesthesiology

Introduction

The paradigm of the 21st century health care system in the USA has shifted, focusing now on value-based driven care with an emphasis on high-quality delivery at lower costs. Health care organizations are tasked with improving the quality of patient care delivery, alongside the more conventional financial aspects of service performance.1 In light of such demands, many health care systems aim to assess the professional performance of their health care workers, with a particular focus on physicians, who remain the primary decision-makers in the care of patients.2 Recently, policy-makers have focused on how various clinical specialties measure and report on the quality of care delivered to patients. Although literature on the subject continues to be produced at an impressive rate, the role of performance feedback on physician improvement and strategies to further involve physicians remains elusive. Furthermore, there are few studies demonstrating positive outcomes, including increased patient satisfaction, improved health outcomes, and decreased costs.3 Examining lessons learned and best practices among current feedback and assessment tools in other fields, however, allows stakeholders to have a better understanding of the importance and strategies of performance feedback.

Hospitals and practices striving to implement performance feedback must consider meaningful and ongoing physician involvement as a crucial cornerstone of care; the physician’s role in redesigning cost-effective quality systems is essential. Consequently, in shaping future delivery systems and payment models for perioperative care, it is imperative that perioperative physicians, including anesthesiologists, are fully engaged in the process.4 Anesthesiologists understand the nuances behind perioperative care and are the health care leaders most appropriate and best positioned to identify quality indicators and their use in conjunction with other performance indicators.

As the USA and other countries move towards an even more data-driven health care medical system, it is critical to learn how best to use such data in the context of improving patient health care.2 Similarly, the many metrics collected during the immediate postoperative period (eg, patient temperature, postoperative pain, nausea) and patient-reported quality of recovery are useful outcome measures, yet are not routinely reported back to clinicians.5 Data collection is meaningless if not used in a constructive manner.

Performance measurement and feedback is most effective when integrated within a broader quality improvement agenda.6,7 Similarly, anesthesiologists and other perioperative physicians are not the only health care professionals who must change and adapt to new roles and expectations when provided with such feedback. For example, anesthesiologists work in the center of a care team and provide collaborative care through a shared decision-making process with surgeons, nurses, and ancillary professional staff members. All members of the care team should be receptive and participatory in creating a culture of transparency and feedback. In performance feedback, it is important to understand that the indirect, albeit most important, beneficiaries of physician improvement are the patients, who remain active participants in the feedback process. The multisource feedback method8 that includes evaluation from patients, peers, and nonphysician workers seems to be the most ideal format for performance assessment in anesthesiologists.

Central to patient satisfaction and the views of evidence-based quality health care, a more robust emphasis should be placed on establishing a formative evaluation process; in this evolution, such assessments should cater for the anesthesiologist, with eventual creation of a valid feedback instrument such as a physician’s individual “report card”. Such a report card should include constant, direct, and timely feedback from quality indicators that are provided to gauge an anesthesiologist’s progress and should ideally include patient-reported, peer-reported, and supervisor-reported measures to target areas for improvement. Embracing these concepts and implementing strategies that reflect the need to improve will aid anesthesiologists in creating both an economically sustainable and high-quality perioperative care system.

In this paper, we aim to provide a narrative overview of the relevant research and evidence supporting the use of quality indicators in medicine, with the focus on perioperative specialties such as anesthesiology. We delve into the concept of performance feedback, highlighting the evidence and identifying how it can be used in a broader quality improvement initiative and as an educational tool. We provide examples of how the principles outlined may be implemented in practice through a discussion of the concept of performance feedback initiatives in the anesthesiology department of a large tertiary care academic hospital. Using the anesthesiologist as an example, this article provides more context on the importance of feeding data back to physicians to improve care, establishes a framework for future research and implementation strategies on performance measurement and feedback for anesthesiologists and other perioperative physicians, and discusses the wider implications for such quality improvement initiatives in and beyond the specialty of anesthesiology.

Tale of medical specialty at the forefront of US health policy

The Institute of Medicine report entitled “To err is human” recognized anesthesia as one of the few health care disciplines that has taken effective action to reduce medical error and improve patient safety.9,10 Yet this was not always the case. The post-World War II era not only signaled economic expansion in the USA, but was also a period when the public began to examine patient mortality more closely. At the time, anesthesiologists took it upon themselves as a medical profession to examine this issue more closely. By the height of the 1970s, it was widely known to the public and the medical community that, as a medical specialty, anesthesiology was at the forefront of patient safety. Oft times, anesthesiology is cited as the medical specialty where patient safety is the central focus that ultimately led to a dramatic improvement in mortality and morbidity rates over the last two decades. While technological advances have certainly played a large role, active engagement of anesthesiologists combined with evolution of their role as perioperative physicians has proven to be of great value in promoting health quality.

The Institute of Medicine report “Crossing the quality chasm: a new health system for the 21st century”, a follow-up to its seminal treatise, further highlighted the importance of patient safety,11 called for a change to the health care system processes to improve the level of quality,12 and explored ways in which such change can be implemented in the health care system. These two significant reports concluded that safety is a quality problem. Furthermore, largely due to burgeoning health care costs that have proven economically unsustainable, the political paradigm and national discourse has shifted to improving quality outcomes while decreasing health care costs. Quality problems are best approached using performance measurements to promote and incentivize improvement.4 Physician performance has emerged as yet another metric aimed at improving health care outcomes and if systematically conducted can help to constrain health care costs.2,3

It is worth noting that while anesthesiologists were once considered the leaders in patient safety, the specialty has largely been absent from the conversation on quality improvement. Perhaps equally as important, anesthesiologists have remained on the proverbial sidelines,13 rather than leading efforts that will dramatically impact payment models and enable better health outcomes.

This process has started to change now that the predominant organization for anesthesiologists in the USA, the American Society of Anesthesiologists, has recognized the importance of engaging the profession at the forefront of such quality improvement initiatives. Through the Anesthesia Quality Institute and the National Anesthesia Clinical Outcomes Registry, the American Society of Anesthesiologists has created a framework for quality benchmarking and outcomes improvement in anesthesiology.2 Assessment of performance is now a core component of anesthesiologist credentialing through the American Board of Anesthesiology.2

In summary, effective quality control requires the specialty to not only develop reliable data collection mechanisms but also put in place systems and processes for effective feedback and use of the data to support quality improvement.14

Quality metrics

Before discussing ways in which feeding back data can be used to improve quality of care, it is important to discuss how quality is defined and measured. Most experts agree with the Institute of Medicine’s definition of quality as “the degree to which health services for individuals and populations increases the likelihood of desired health outcome and are consistent with current professional knowledge”.1

Modern day quality improvement initiatives benefit from the earlier work of the Donabedian structure–process–outcome conceptual model, which centers on the relationship and proximity of a variable to the desired performance result.15 While a comprehensive primer and broader overview of the rationale and science of performance measurement is beyond the scope of the present paper and has been well reported elsewhere,2 it is worth mentioning a few topics to provide the framework for this discussion.

The predominant model of data use within the quality improvement strategy is based on process measures. Process measures describe what is done to patients, eg, administering antibiotic prophylaxis within one hour of surgical incision.2 Care processes are monitored continuously for process changes that are rapidly detectable.7 Unlike outcome measures, process measures are directly actionable, do not require risk adjustments when the target population is appropriately selected, and do not measure quality for most activities.2 From an intervention and education standpoint, it is generally easier to target process measures instead of outcome measures.

Next, there are structure measures that reflect the properties of the hospital setting, and typically include organizational characteristics, human resources, and technology, eg, adequacy of facilities, qualifications of medical staff, procedure volume, health information technology.16,17 Structure measures are relatively inexpensive and easy to collect, and procedure volume, especially with certain operations, has a strong correlation with outcome.18 These structure measures do not assess the performance of individual physicians and while considered especially important by regulatory agencies and third-party payers, particularly as a way to incentivize adoption of organizational processes considered best practices, they will not be discussed further.2,9

Lastly, there are outcome measures, eg, postoperative nausea/vomiting and complications, that represent “the common product of all clinical activity and the cornerstone of performance measurement”.2 Performance measurement is associated with improved outcomes. The type of data (less expensive, less accurate administrative data versus the more expensive, more accurate clinical data), data quality, risk adjustment, and sample size or frequency of the outcome are some of the key factors affecting how outcome measures can be used, particularly when measuring physician performance. There does seem to be a strong association between outcomes feedback and improved population outcomes. Outcomes did improve when physicians received feedback specifically on outcome data.2

Concept of performance feedback

Before defining the key term of this paper, it is worth distinguishing feedback from commonly interchangeable yet very distinct terms, such as encouragement and evaluation. Encouragement (eg, “great job”) is purely supportive and does very little to improve an individual’s skills. Evaluation is often summative and represents the final judgment of an individual’s performance.19 Perhaps the most distinguishing characteristic among these similar terms centers on the inherent design of feedback, ie, to improve future performance.

Feedback is often described as the act of providing knowledge of the results of behavior or performance to the individual.20 Within health care, feedback is “any summary of clinical performance of health care over a specified period of time, given in a written, electronic or verbal format”.7,21 However, feedback is simply more than just providing information, and must incorporate some action to close the identified gap and promote improvement.22 Another definition that reflects these points and is the one preferred for the purposes of this paper is “feedback is an informed, nonevaluative, objective appraisal of performance intended to improve clinical skills”.23,24

From an organizational standpoint, feedback over time can be an important mechanism for organizational learning resulting in both “incremental and large-scale modification to care systems and processes” over time.2

Performance feedback in the medical community

There is a wide variety of stakeholders in the health care system focused on improving physician performance (eg, medical groups, professional medical societies, purchases/employers, private sector companies, health plans, and the federal government). The multiple efforts aimed at reporting performance data to individual physicians vary in terms of objectives. However, these stakeholders share the underlying goal of making physicians aware of their performance and encouraging improvement in specific aspects of care delivery, such as clinical quality, patient experience, patient safety, and resource use, when performance is lacking.6

Physician-level performance measurement and feedback is a relatively new concept. The experiences that organizations or even specialties within medicine have had in measuring performance and providing feedback to individual physicians are fairly limited.6 Important lessons learned in applied settings often remain unknown to others with an interest in using physician performance results to change physician behavior.

While performance measurement is an imperfect science, it is undoubtedly becoming a mainstay in the national quality improvement agenda. Future research is required to determine how best to harness the potential role it has in promoting and incentivizing improvement in the quality of health care.

Effectiveness of feedback performance

While feedback performance is reported to make a moderate difference in outcome measures depending on the overall quality improvement agenda incorporated, the evidence for its effectiveness is limited at best. Unfortunately, there are few randomized controlled studies beyond anecdotal examples that validate the use of this tool even though it has been extensively researched. Still, it warrants further discussion. Three research studies are examined to explore this topic.

Over 25 years ago, Tierney et al conducted a randomized controlled trial to compare the effects of supplying monthly feedback reports of compliance with preventive care protocols by 135 internal medicine house staff with the effects of providing specific reminders at the time of patient visits.25 The study concluded that physician compliance with suggested preventive care protocols can be increased by both delayed feedback and immediate reminders. The data also strongly suggested that feedback reports are effective strategies to promote better and more responsible quality care.

The next study examined incorporated benchmarking to increase the overall effectiveness of performance feedback to clinicians. This 2001 study, which was part of the Ambulatory Care Quality Improvement Project in Alabama evaluated the effectiveness of using achievable benchmarks to enhance typical physician performance and improve care.26 In this prospective cohort study, physicians were randomly assigned to receive a multimodal improvement intervention that included physician-specific feedback or an identical intervention plus achievable benchmark feedback.27 The main outcome measures were changes in the proportion of patients receiving routine care (ie, a diabetic having a foot examination), vaccination (eg, influenza vaccine), and blood tests (eg, glucose control, and cholesterol and triglyceride levels) when compared between the two groups. Overall, the results demonstrated that use of achievable benchmarks significantly enhances the effectiveness of physician performance feedback in the setting of multimodal quality improvement intervention. It is important to recognize that audit and feedback methods, whereby clinicians receive reports of their performance are compared with the mean performance of a peer group, have been used and studied extensively but few of these studies have been in the form of randomized controlled trials. Still, clinicians care about their performance especially relative to their peers. The underlying theory is that viewing personal performance within the context of peer performance is a powerful motivator for change. However, there are only modest benefits with limited long-term sustainability. Hence, seeking a method to increase the effectiveness of using performance feedback for anesthesiologists may require an achievable benchmark method that can be developed from quality indicators, much like the ones currently being developed by the Anesthesia Quality Institute.28

The concept of feedback performance is not unique to the field of medicine. From the aviation industry to Wall Street, feedback performance is an important part of quality improvement. A comprehensive literature search was done reviewing various types of feedback interventions since the early 1900s. Many interesting facts were noted, including the inconsistent results concerning the effectiveness of feedback. Some early experiments found that feedback improved performance for some indicators and impaired performance for others. In the 1950s, after an extensive review of the literature, Ammons, a pre-eminent scholar on feedback, concluded that feedback increased learning and motivation.30,31 His studies were not met without controversy, as it was later discovered that he failed to report the findings that contradicted his conclusions. Still, the positive effects of feedback on performance became one of the widely accepted principles in psychology. Next, over 131 papers were reviewed for meta-analysis and the results indicated a modest but positive effect of feedback on performance overall (less than one-half of one standard deviation improvement in performance), but 38% of feedback effects were negative, giving a very inconsistent picture of feedback effectiveness.19 Table 1 summarizes a few major papers addressing performance feedback.

Table 1 Summary of major findings for effectiveness of feedback and performance measurement

Barriers and quality indicators

Feeding back data in anesthesiology and other perioperative specialties is complex. It is difficult to ascertain which process or outcome measures are anesthesia-related. Optimal care of the surgical patient requires a multidisciplinary team approach consisting of the unique integration of anesthesiologists, surgeons, other proceduralists, and nurses. In a systematic review of the anesthesiology literature, 108 quality indicators were identified; 57% were outcome measures, 42% were process measures, and 1% involved structure of care.4 Despite this, there is no validated algorithm to identify anesthesia-related outcomes.2 Ideally, quality indicators should be transparent, reliable, evidenced-based, measurable, and improvable.7 The quality of feedback is predicated on this notion.

Hesketh et al identified a number of barriers to giving effective feedback. These include feedback that is listed as too generalized and not related to specific facts or observations, inconsistent feedback from multiple sources, feedback that is not actionable (ie, not providing guidance on how to rectify behavior), and a lack of trust or respect for the source of feedback.33 Physicians are more influenced by an authoritative, credible source.34 Van der Veer et al identified lack of trust in data quality, lack of intensity of feedback, and lack of motivation.24,27

Perioperative outcomes in anesthesia

Mortality due to anesthesia is no longer the pervading outcome it once was over 40 years ago. Risks of perioperative mortality related to anesthesia are now less than an estimated 0.2% in healthy patients.13,36 Mortality itself is now considered a poor quality indicator because it is rare and often due to other factors beyond an anesthesiologist’s care.7 Still, there are other routine outcome measures affected by variation in anesthesia care worth considering. Contemporary medicine has widely demonstrated that patient outcomes are multifactorial. Outcomes are not solely a result of the effectiveness of medical care but are also affected by a complex interplay between patient’s risk factors and random events.37 Complications such as patient awareness under general anesthesia, neurological deficit after regional anesthesia, or even an epidural abscess after neuroaxial anesthesia may be more appropriately linked to anesthesia. Risk adjustment, the concept of not penalizing hospitals and physicians who treat high-risk patients,38 is an important factor affecting outcome measures. When generating unbiased, constructive quality reports from the variable nature of patient outcomes, risk adjustment plays a major role.

Postoperative recovery and patient satisfaction

Variation in the quality of anesthesiology care may be more directly assessed in the postoperative period.7 A patient’s recovery in the postoperative care unit is closely linked to the quality of anesthetic technique and medications given to minimize postoperative issues, such as nausea and vomiting and pain, two of the most important factors affecting patient’s experience, quality of recovery, and overall satisfaction with care.8,39 Postoperative nausea and vomiting and pain have a strong negative effect on patient satisfaction.23 These two undesirable complications represent important quality indicators for the anesthesiologist. Pain can be measured using a variety of scales (eg, visual analog scale, numerical rating scale) that can be relayed back to the anesthesiologist. The nursing staff in the post anesthesia care unit monitors other quality measures that are elements of the patient’s experience. Data such as temperature upon arrival, time spent in the recovery room, and if a day surgery patient, the time to void and ambulate are quantifiable and can be used in the feedback process.7

Patient satisfaction surveys involve quantification of a patient’s subjective perceptions. There are a number of surveys that have been developed and validated, and more importantly, can be used as part of the anesthesiologist’s performance profile. The American Society of Anesthesiologists is currently working to define valid, reliable, and useful clinical quality indicators for anesthesiologists. Identifying the appropriate process and outcome measures is the first requisite. Afterwards, the data can be turned into information that is both useful and actionable. Later, this can be implemented through a feedback mechanism.

There are various feedback strategies that yield a range of results. The evidence overall suggests that providing feedback results in small to moderate positive effects on clinical practice.7 There are several possible factors that help with providing feedback. A study by Van der Veer et al found that success factors included timely dissemination of feedback from data collection and presenting information in a confidential and nonjudgmental manner.24 Mugford et al showed that information feedback can influence clinical practice if the information was presented close to the time of decision-making and particularly when physicians had agreed beforehand to review their practice.35 Jamtvedt et al identified source and duration of feedback as important characteristics. Parsloe et al identified that feedback must be given sensitively and appropriately.40,41 This study also commented on other aspects involving the person giving the feedback and the recipient, including differences in sex, age, and educational or cultural background.27 These may not be viewed as obstacles, but can make feedback sessions strained and demotivating. De Vos et al established that feedback reports within an educational implementation strategy were most effective.42 Barriers to quality improvement identified were lack of credible data, lack of supportive local management, and lack of hospital resources.43 Effective feedback is continuous, timely, credible, nonpunitive, and supportive of remedial action.44

Case example: monitoring and feedback in anesthesiology

Performance measurement and feedback has broader implications for the future of medicine. In particular, timely integration of feedback facilitates early and sustainable adoption of quality improvement initiatives. Moreover, integrating a systematic feedback process earlier on in medical training will likely encourage a culture where feedback is routine and part of the norm. While the direct educational merit of performance feedback within residency training programs is outside the scope of this paper, it is worth discussing, giving the example provided from a large tertiary academic medical center.

The anesthesiology residency program at our institution, Brigham and Women’s Hospital, comprises approximately 90 residents, 30 fellows, and 110 faculty members. In efforts to increase the evaluation process and provide residents with a more meaningful and comprehensive feedback tool, the department recently implemented a multisource feedback program for residents. Multiple international studies have used multisource feedback or 360 degree evaluation, which is an increasingly popular way of assessing multiple components of professional performance.35 This rather reliable and valid feedback method uses external evaluation of physician performance and focuses on various tasks by involving: supervisors (such as faculty); peers with knowledge and experience of scope of practice; nonphysician coworkers (nurses, allied health care professionals, or administrative staff); and patients.35,36 Raters in each category who observed the physicians’ behavior are asked to answer questions about the physicians’ performance and this was compared with the physician’s self-evaluation to examine directions for change.36

Although on the surface the decisions of key people on the higher rungs of the ladder in medicine make the more directed decisions of a person’s care, medicine has always been and is increasingly becoming team-based. For a large organization, such as a medical center or even a community hospital that sees multiple people from different departments playing a variety of different roles, the 360-degree performance appraisal may be beneficial and a reinforcement of this team model concept.

This multisource method gathers anonymous information from peer residents (and/or colleagues), ancillary staff, nurses, social workers, and anyone who has the opportunity to interact or observe the individual on a regular basis. Ideally, this should supplement and better inform a superior’s own observations. This should be a fair, consistent, and, most importantly, a private affair to ensure the integrity of the process. The supervisor (program director for residents, chair of the department, or medical director for attending) should directly ask the individual who is the source of the evaluation if clarifications of comments are needed, especially in situations where improvement is needed.

While it is both practical and logical to tie financial incentives to those for exemplary performance reviews, the constraints within an academic center may make this very difficult to achieve. Awards and recognition programs are a good way to not only boost department morale but also entice most if not all staff members to participate.6

The future: integrating feedback platform into practice

Feedback is an important component of the ongoing development of practitioners in health care. For physicians in particular, feedback is central to developing both confidence and competence at all stages of their medical careers. Clinical practice, professional behaviors, and attitudes are routinely assessed using a variety of workplace instruments (eg, multisource feedback, case-based discussions, such as mortality and morbidity conferences). An example of this was discussed earlier in a case example from a residency training program in a tertiary center.

There are a few goals that need to be achieved when developing and implementing feedback initiatives, including solid cultural and financial infrastructures. Indeed, the concept of feedback needs to be readily embraced and encouraged by senior members of the clinical department and other stakeholders in the hospital at large who feel that this initiative is important. The “buy-in” includes the expectation and view that feedback occurs on a routine basis and those engaged in the process are both encouraged to participate and be held accountable. A warranted concern from those giving feedback is the potential negative consequences resulting from providing constructive feedback, fearing the relationship with others may be damaged.38

Feedback must be part of an overarching quality improvement agenda; it is not meant to be an isolated, fragmented initiative that is typically undermined by lack of resources or systemic barriers to improvement within programs (Table 2). In addition to the financial investment and human resources required, the issue of inadequate training can be addressed by faculty development activities that allow participants to try out new skills, receive feedback on their performance, and network with peers to exchange ideas and best practices. Lack of training can also be addressed with self-study around the topic of feedback. Establishing feedback as a regular and recurring event focused on increasing patient satisfaction and outcomes may prevent defensive reactions on the part of opponents.

Table 2 Major premises of integrating feedback and establishing a culture for physician improvement

Effective feedback should be an integral part of clinical practice. While barriers to feedback for anesthesiologists and other perioperative physicians exist, these concerns can and will be addressed. The American Society of Anesthesiologists and other key stakeholders are identifying specialty specific indicators that can be used when creating a broader quality improvement agenda. Placing a more immediate, formal feedback strategy that focuses on goal-oriented behavior is rapidly becoming a mainstay. Ultimately, this will improve patient care. In the future, as a summative performance assessment report is created, it is envisioned that physicians may use their individual feedback information for reflection and designing personal development plans as lifelong learners and leaders in improving patient care.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

Jencks SF, Wilensky GR. The health care quality improvement initiative: a new approach to quality assurance in Medicare. JAMA. 1992;268:900–903.

2.

Moya-Ruiz C, Peiró S, Meneu R. Effectiveness of feedback to physicians in reducing inappropriate use of hospitalization: a study in a Spanish hospital. Int J Quality Health Care. 2002;14:205–312.

3.

Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423–1431.

4.

Glance LG, Fleisher LA. Anesthesiology and health care reform: a call to action. Manuscript draft.

5.

Benn J, Arnold G, Wei I, Riley C, Aleva F. Using quality indicators in anaesthesia: feeding back data to improve care. Br J Anaesth. 2012;109:80–91.

6.

Kiefe CI, Woolley TW, Allison JJ, Box JB, Craig AS. Determining benchmarks: a data driven search for the best achievable performance. Clin Perform Qual Health Care. 1994;2:190–194.

7.

Kiefe CI, Allison JJ, Williams OD, et al. Improving quality improvement using achievable benchmarks for physician feedback. JAMA. 2001;285:2871–2879.

8.

Ende J. Feedback in clinical medical education. JAMA. 1983;250:777–781.

9.

Institute of Medicine. Crossing the quality chasm: a new health system for the twenty-first century. Washington, DC, USA: National Academy Press; 2001. Available from: http://www.iom.edu/~/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf. Accessed March 28, 2013.

10.

Chassin MR, Galvin RW; National Roundtable on Health Care Quality. The urgent need to improve health care quality. JAMA. 1998;280:1000–1005.

11.

Glance LG, Neuman M, Martinez EA, Pauker KY, Dutton RP. Performance measurement at a “tipping point”. Anesth Analg. 2011;112:958–966.

12.

Telek SS, Shaw R, Damberg CL, McGlynn EA. Providing performance feedback to individual physicians: current practice and emerging lessons: final report. Santa Monica, CA, USA: Rand Corporation; 2006. Available from: http://www.rand.org/pubs/working_papers/WR381. Accessed March 30, 2014.

13.

Donabedian A. Criteria, norms and standards of quality: what do they mean? Am J Public Health. 1981;71:409–412.

14.

Donabedian A. Evaluating the quality of medical care: 1966. Milbank Q. 2005;113:1004–1006.

15.

Shute VJ. Focus on formative feedback. Rev Educ Res. 2008;78:153–189.

16.

Steiner DL, Norman GR. Health Measurement Scales: A Practical Guide to their Development and use. 4th ed. Oxford, UK; Oxford University Press; 2008.

17.

Findlay G, Goodwin A, Protopapa K, Smith N. Knowing the Risk: A Review of the Perioperative Care of Surgical Patients. London, UK: National Confidential Enquiry into Patient Outcome and Death; 2011.

18.

Haller G, Stoelwinder J, Myles P, McNeil J. Quality and safety indicators in anesthesia: a systematic review. Anesthesiology. 2009;110:1158–1175.

19.

Hesketh EA, Laidlaw JM. Developing the teaching instinct. 1: Feedback. Med Teach. 2002;24:245–248.

20.

Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29:642–647.

21.

Hutchinson A, Young TA, Cooper KL, et al. Trends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning System. Qual Saf Health Care. 2009;18:5–10.

22.

Iezzoni LI. Reasons for risk adjustment. In: Iezzoni LI, editor. Risk Adjustment. Chicago, IL, USA: Health Administration Press; 2003.

23.

Ramaprasad A. On the definition of feedback. Behav Sci. 1983;28:4–13.

24.

van der Veer S, de Keizer N, Ravelli A, Tenkink S, Jager K. Improving quality of care. A systematic review of how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79:305–323.

25.

Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24:659–666.

26.

Hayes RP, Ballard DJ. Review: feedback about practice patterns for measurable improvements in quality of care – a challenge for PROs under the Health Care Quality Improvement Program. Clin Perform Qual Health Care. 1995;3:15–22.

27.

Allison JJ, Kiefe CI, Weissman NW. Can data-driven benchmarks be used to set the goals of Healthy People 2010? Am J Public Health. 1999;89:61–65.

28.

Anesthesia Quality Institute. Available from: https://www.aqihq.org/. Accessed April 14, 2014.

29.

Overeem K. Doctor performance assessment: development and impact of a new system. Perspect Med Educ. 2012;1(2):98–100.

30.

Ammons RB. Effects of knowledge of performance: a survey and tentative theoretical formulation. J General Psychology. 1956;54:279–299.

31.

DeNisi AS, Avraham NK. Feedback effectiveness: can 360-degree appraisals be improved? Academy of Management Perspectives. 2000;14:129–139.

32.

Veloski J, Boex JR, Grasberger MJ, Evans A, Wolfson DB. Systematic review of the literature on assessment feedback and physicians’ clinical performance: BEME Guide No 7. Med Teach. 2006;28:117–128.

33.

Hesketh EA, Anderson F, Bagnall GM, Driver CP, Johnston DA, Marshall D, Needham G, Orr G, Walker K. Using a 360 degrees diagnostic screening tool to provide an evidence trail of junior doctor performance throughout their first postgraduate year. Med Teach. 2005;27(3):219–233.

34.

Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2006;2: CD000259.

35.

Mugford M, Banfield P, O’Hanlon M. Effects of feedback of information on clinical practice: a review. BMJ. 1991;303:398–402.

36.

Iezonni LI. The risks of risk adjustment. JAMA. 1997;278:1600–1607.

37.

Whitty P, Shaw I, Goodwin D. Patient satisfaction with general anaesthesia. Anaesthesia. 1996;51:327–332.

38.

Heidegger T, Husemann Y, Nuebling M, et al. Patient satisfaction with anesthesia care: development of a psychometric questionnaire and benchmarking among six hospitals in Switzerland and Austria. Br J Anesthesia. 2002;89:863–872.

39.

Gigante J, Dell M, Sharkey A. Getting beyond “good job”: how to give effective feedback. Pediatrics. 2011;127:205–207.

40.

Parsloe E, Wray M. Coaching and Mentoring. London, UK: Kogan Page; 2000.

41.

Parsloe E. Coaching, Mentoring and Assessing. London, UK: Kogan Page; 1995.

42.

De Vos M, Graafmans W, Kooistra M, Meijboom B, Van Der Voort P, Westert G. Using quality indicators to improve hospital care: a review of the literature. Int J Qual Health Care. 2009;21:119–129.

43.

Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. Data feedback efforts in quality improvement: lessons learned from US hospitals. Qual Saf Health Care. 2004;13:26–31.

44.

Overeem K, Wollersheim HC, Arah OA, Cruijsberg JK, Grol RP, Lombarts KM. Evaluation of physicians’ professional performance: an iterative development and validation study of multisource feedback instruments. BMC Health Serv Res. 2012;12:80.

Creative Commons License © 2014 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.