Applying Expectancy Theory to residency training: proposing opportunities to understand resident motivation and enhance residency training

Background

Medical resident education in the United States has been a matter of national priority for decades. In 1972 the Liaison Committee for Graduate Medical Education was established, then superseded in 1981 by the founding of the Accreditation Council for Graduate Medical Education (ACGME).¹ One of the ACGME’s major initial directives was to address the variability in the quality of medical resident education across the country.^1,2 A focus on program structure then ensued that was designed to address this problem.¹ Furthering this approach, in 1999 the ACGME introduced six well-defined national domains of residency training competency, namely Patient Care, Systems-Based Practice, Medical Knowledge, Practice-Based Learning, Professionalism, and Communication.^1,3

The effects of this national formalization in educational aim and structure were “largely salutary”.¹ This success nevertheless did come at a cost, with the structure of resident educational programming being emphasized. As reviewed in a Special Report in the New England Journal of Medicine, resident educational programs had become “prescriptive”, while innovation in education remained absent.¹ In response to this limitation, a 2009 national restructuring of the ACGME accreditation system was developed, the Next Accreditation System.¹ The contemporary focus of ACGME accreditation was to be outcomes-based; ie, the attainment of “Milestones” in the prior established six competencies. This change in methodology aimed for “the introduction of innovation through accreditation”.¹

During the approximate time period above, synchronous concerns were being raised in the surgical education community. Myriads of stressors to surgical residency training were noted. The 2002 presidential address of Dr Haile T Debas to the American Surgical Association (ASA) reflected on these issues,⁴ and in response, the ASA convened a Blue Ribbon Panel to review a broad range of issues of concern. Participants included representatives from the American College of Surgeons (ACS), the American Board of Surgery (ABS), the Residency Review Committee-Surgery (RRC-S), as well as the ASA. The committee:

[…] was charged with […] making recommendations regarding the changes needed in surgical education to enhance the training of surgeons to serve all the surgical needs of the nation, and to keep training and research in surgery at the cutting edge in the 21st century.⁵

In all, the view was held that:

to retain its leadership position in innovation and its attractiveness as a career choice for students, surgery must evolve with the times […] surgery needs to introduce changes to create new priorities in clinical practice, education, and research […].⁴

Further supporting a needed reevaluation of contemporary surgical residency educational practices, the ABS and the ACS established standing committees on resident education. Stated aims involved: addressing issues of resident education in surgery, including the educational environment, characteristics of the resident population, graduate surgical education curricular issues and competency, and faculty development as educators.^6,7

In sum, the recognition of prior limitations in graduate medical education including surgical residency training specifically have brought to bear a contemporary approach to education. This approach is now directed at ensuring quality in learning as well as in patient care.¹

Brief overview on work-motivation theories

Work-motivation theories attempt to explain people’s choice, persistence, and performance in tasks. Specifically, these theories explicate the factors that influence individuals’ motivation toward achievement.^8–18 A detailed discussion of work-motivation theories is beyond the scope of this article, yet organizing comments are possible. Several theories have been postulated. Expectancy-value theorists posit that selection and motivation in tasks are explained by 1) expectancy: one’s perception of how well they will perform, and 2) value: the value placed on the endeavor chosen.^10–12 Bandura further delineated expectancies as either efficacy- or outcome-based: the former a belief that one can accomplish the chosen task; the latter a belief that particular actions will lead to a given outcome.¹³ Importantly, shared amongst achievement motivation theories is the hypothesis that beliefs in one’s abilities are significant factors in one’s motivation. Positive beliefs are argued to have positive effects with negative beliefs having negative effects.^14,15 As a corollary, self-determination theorists opine that desires for continued competence motivate persons to pursue further challenges.^16,17 In sum, work-motivation theorists explain individuals’ actions as related to one’s expectations, valuations, and perceptions as well their desire for continued competence.^8–18

Expectancy Theory

Expectancy Theory is well known in the work-motivation literature. It was first proposed by Vroom after studying the motivations behind individuals’ work-related decision making.¹⁸

Current residency training in the United States is in many ways analogous to the traditional workplace. Varied work assignments, interactions with diverse personnel, workplace hierarchy, time constraints, performance review, and professional advancement are examples. The analogies between residency and the workplace provide an innovative opportunity to apply work-motivation theory to residency training. The aim: to enhance medical education. Similar to other work-motivation theories, Expectancy Theory holds that a person’s choice in certain action reflects a belief that such action will result in a desired outcome. Expectancy Theory is divided into three components: Expectancy, Instrumentality, and Valence. As proposed by Vroom,¹⁸ Expectancy Theory involves an interaction between these three components, monotonically summing in a Motivational Force. Namely, the functional value of one’s Motivational Force will increase as the value of each variable, Expectancy, Instrumentality, and/or Valence, increases.

With its three components, Expectancy Theory appears particularly relevant as a work-motivation theory relating to residency education. Its application affords new insights into residents’ motivation in training. Such perspectives could advance innovative changes in residency training.

Expectancy

As Expectancy Theory involves the interaction of Expectancy, Instrumentality, and Valence, we will explicate each component, then follow with suggestions in applying it to residency training. Expectancy is the expectation that one’s efforts be associated with achieving one’s goals. In this way Expectancy denotes an action-to-outcome process: the subjective probability that a particular act will lead to a particular outcome.¹⁸ Moreover, Expectancy links degrees of effort to probabilities of outcome. In practice, what is manifested is an association between effort and outcome; a “motivation for effective performance”.¹⁸ Expectancy Theory holds that a strongly perceived connection between one’s efforts and one’s outcomes will enhance motivation; ie, “[…] individuals that believe that greater levels of effort are certain to result in success whereas lesser levels of effort are certain to result in failure […] will choose to exert the greater amount of effort”.¹⁸ Supporting this perception is the finding that when workplace promotions are based on strong performance, high Expectancy follows; whereas promotions based on nepotism provide lower Expectancy.¹⁸

In sum, according to Expectancy Theory, an individual’s motivation will be increased, if it can be established that one’s efforts will lead to one’s goals. Moreover, motivation will be further increased if even greater efforts percept as joining to greater success.

Expectancy in application to residency training

Residency training entails myriads of paradigms with the potential to apply Expectancy. Yet in our view such paradigms currently transpire without their fully realized potential to enhance resident motivation. For example, it is a longstanding tradition in surgical training that resident preoperative preparation facilitates maximal resident case involvement. Yet typically, formative assessments do not occur preoperatively, thereby not enabling this desired interaction between resident and surgeon. Applying Expectancy establishes a formative preoperative discourse, affording an enhanced educational opportunity for both that case (via greater involvement), as well as for future cases (via greater motivation for subsequent case preparation).

Training in the intensive care unit (ICU) reflects another opportunity to apply Expectancy in resident education. Here, residents of varied skill levels provide varied levels of care to complex patients. Historically, when recognized by attending physicians, increased resident patient-care competence is acknowledged by greater autonomy in subsequent resident-directed ICU care. Importantly however, “real-time” formative assessments of resident performance during their ICU rotation are typically lacking. More commonly, post-ICU rotation (remote) summative evaluations are effected by teaching staff. From a perspective of Expectancy Theory, the current paradigm does not maximize the potential for increasing resident motivation. Expectancy posits that if frequent formative resident evaluations occurred during the ICU rotation, a more continual progression toward resident autonomy would occur, thus enabling greater advancement in resident competence. Expectancy explicates that such a continual action-to-outcome process, ie, work, to recognition, to acknowledgment, would increase resident motivation in education.

Importantly, the training scenarios described above are well established in residency training (eg, resident operative case involvement and ICU care). Yet what we propose is a newly depicted formative process in such education. Namely, that resident actions be more clearly delineated as connected to subsequent educational outcomes in “real-time”. In applying this new method to traditional training paradigms, advances in residency practice could be created.

Of note, separate to the above, the lens of Expectancy Theory also provides new opportunities to view other historic aspects of residency training, eg, task-oriented endeavors. Typically, such “service work” is viewed by residents as onerous and of less educational value. Moreover, current training practice does not typically afford formative recognition of an individual resident’s “service” efforts. Rather, after-the-fact standardized, predominantly “check-box”, evaluations are completed. Yet Expectancy Theory proposes that if a resident’s daily efforts in task-oriented endeavors were formatively recognized, residents would associate one’s personalized efforts with individualized success. According to Vroom,¹⁸ this increased Expectancy would monotonically increase Motivational Force. It is noteworthy that the detrimental effect of “service-work” has been reported in a study on attrition in residency programs, whereby residents described being “overwhelmed by job responsibilities”.¹⁹ This negative effect was also summarized in the presidential address of the ASA a decade ago: “[…] their perception that the life of the surgical resident is stressful, the work hours too long, and the time for personal and family needs inadequate”.⁴

Thus, applying Expectancy Theory to residency training offers both innovative opportunities to enhance resident motivation, as well as to mollify long-held, traditionally negative views in training.

Regarding contemporary issues in residency education, Expectancy relates well to simulation training in medical education. One illustration is the Fundamentals in Laparoscopic Surgery (FLS) course now broadly sponsored by the ACS as well as the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES).²⁰ Completion of this laparoscopic maneuvers course enables improvement in one’s laparoscopic skills; this is validated in several publications to date.^20–25 We propose that an explanation for the course’s success rests in more than a student’s mere attainment of the mechanics of the technical maneuvers. Through the lens of work-motivational theory and Expectancy, the FLS course effects an action-to-outcome process, ie, facilitating students’ sequential appreciation of their now advancing laparoscopic skills. Stated otherwise, Expectancy posits the FLS course as providing residents with direct, visualized feedback of their progressing surgical skill, making readily tangible the “fruits” of their labor.

Further applying Expectancy to simulation training, we note the live animal (porcine) model of the Advanced Trauma Operative Management (ATOM) course.²⁶ In this ACS-sponsored course involving various traumatic injuries, participants are first given didactic instruction from experienced surgical mentors. What follows is “hands-on” operative management of such injuries. Via this two-stage format, a positive learning experience results, as depicted in residents’ heightened expectation of future success in trauma management.^27–30 This change in residents’ perceived ability to manage future human-trauma patients (ie, pre-course compared to post-course confidence) appears directly linked to one’s participation in the animal-lab ATOM course.^27–30 As depicted above for the FLS course, this finding can be explicated by an action-to-outcome perspective. Expectancy proposes that the didactics (learning about injuries), immediately followed by the operative intervention itself (repairing such injuries), produces a strongly motivational action-to-outcome process. As viewed by Expectancy Theory, such perspectives on the educational effect of simulation training support the expanded use of such programs.

In review of Expectancy applied to residency training, we posit that numerous educational opportunities exist whereby action-to-outcome processes could be potentiated.

Instrumentality

The second parsed component of Expectancy Theory is Instrumentality. This reflects the motivational effect created when an anticipated reward is subsequently given.¹⁸ Several factors are associated with Instrumentality, namely trust in one’s superiors; control in the performance-to-reward process; and the presence of policies relating performance and rewards.¹⁸

Trust in one’s superior stems from a twofold belief by junior personnel that: 1) greater individualized efforts will be noted by superiors; and 2) anticipated rewards will be actualized. Ultimately, the greater the trust a subordinate holds in their superior, the greater the perceived Instrumentality.¹⁸

Control in the processes of performance and reward is the second factor in Instrumentality.¹⁸ A sense of control arises with the perception that one’s enhanced efforts will be registered by others, and will lead to greater rewards.¹⁸ The greater one’s perceived control in the performance and reward process the greater the Instrumentality.¹⁸

Lastly, the presence and recognition of formalized policies impacts Instrumentality.¹⁸ Well-delineated metrics are common prerequisites in performance-related policies. These policies then direct merit-based rewards.¹⁸ According to Expectancy Theory, an individual’s internalization of such policies best facilitates reaching performance metrics. In effect, the mere knowledge of a performance-related policy can bring greater effort toward achieving it.

Instrumentality in application to residency training

As noted for Expectancy, Instrumentality also appears highly relevant to residency education. First, residency encompasses daily, continuous interaction between junior and senior personnel. These interactions enable numerous performance efforts to be formatively acknowledged. From the perspective of Instrumentality, such formative acknowledgments should lead to the development of trust between junior and senior personnel. This increased trust then creates greater Instrumentality and motivation.

As residency training involves many hours of work in multitudinous tasks, countless opportunities exist for residents to individualize their performance efforts. Acknowledgement by superiors of their junior personnel’s individualized efforts creates a sense of control for junior residents. Namely, the harder one chooses to work, the greater the recognition for one’s efforts. As proposed by Instrumentality, such awareness links workplace efforts to workplace motivation.¹⁸

Additional opportunities regarding Instrumentality and residency relate to metric-based training paradigms. Many such examples exist, including: expectations in work efficiency; expectations of work volume; competence in patient-care rounds; preparation for operative cases; attention to interpersonal interactions; professionalism in patient and family communication; cost-efficient use of diagnostic tests/resources; and dexterity with procedure/surgical-related skills. Yet, current residency practice does not typically have formal reviews with residents at the start of assigned rotations about the performance metrics by which they will be graded. More typical is a “learn the service as you go approach”. Moreover, evaluations of resident performance are completed after the fact (remotely) and predominantly via “check-box” summations. Expectancy Theory and Instrumentality denote these as missed opportunities for increasing resident motivation. Ensuring residents’ awareness of specific performance thresholds at the outset is proposed. According to work-motivation theory/Expectancy Theory, residents should then exhibit greater motivation in meeting such educational metrics. Of note, further supporting such use of Instrumentality in residency training is the ongoing research study, MSU GOAL (Guided Operative Assessment Learning), directed by the Michigan State University Department of Surgery. In testing an Instrumentality-type approach, this study is assessing whether formatively defining a specific surgical skill, before the start of an operative case, will then facilitate greater motivation toward teaching and learning that skill during the case. This hypothesis is being quantified via an electronic questionnaire assessment tool, completed separately by both resident and educator, after the case.³¹

In addition to individual institutional training metrics, more standardized national residency metrics also exist relevant to Expectancy Theory and Instrumentality. Notable examples are the ACGME’s six national core competencies and the more contemporary Milestone Project.^1,3 The latter denotes collaboration between the ACGME and subspecialty organizations in developing well-defined sequential training milestones assessing resident competency.^1,3,31 From a perspective of work-motivation theory and Expectancy Theory, elucidating such metrics affords motivating opportunities for residents to aspire to such goals. In fact, the ACGME Milestones Project is a strong step forward in support of an Instrumentality approach to medical education. And yet, despite this newly defined and more specific set of enunciated metrics, we posit their not-fully realized potential in motivating residents, as residents are neither sufficiently aware nor are they frequent enough recipients of formative evaluations regarding them. Thus, from a perspective of Instrumentality, we posit that in formalizing resident recognition of metrics, and in formatively reviewing resident progress toward each aim, significant opportunities exists to increase Instrumentality and enhance motivation in medical education.

Of note, examples of national, specialty-specific metrics in residency training exist as well. The delineated modular curriculum of the consortium of the Surgical Council on Resident Education (SCORE) is one example.³² Importantly, SCORE compromises comprehensive surgical education representation; namely the ABS, ACS, ASA, RRC-S, the Association of Program Directors in Surgery (APDS), as well as the Association for Surgical Education (ASE). Established in 2006, “[…] with responsibility for resident education in surgery and an interest in improving the training of surgeons”,³² SCORE aimed to be a widespread, competency-based curriculum with specified “desired outcomes”.³² One particular example within this metric-based curriculum is the satisfactory completion by residents of a surgical skills laboratory. Skills include asepsis technique, instrument manipulation, suturing, advanced tissue handling, airway management, endoscopy, laparoscopy, and stapled anastomoses.³² Through the lens of Expectancy Theory, we posit SCORE as a currently not-fully realized opportunity to enhance surgical resident motivation in training. Typically, resident completion of a SCORE module is recorded quantitatively, without detailed qualitative review. Yet Instrumentality explicates that a qualitative, formative recognition would bring greater resident motivation in subsequent SCORE modules. Thus in applying Expectancy Theory, it is suggested that program emphasis shift away from the mere recognition of completed SCORE tasks, moving toward more formative individualized reviews of one’s performance, within each SCORE module.

In sum, as viewed through a lens of Expectancy Theory, current metrics in residency training do not appear to function with maximal resident motivational effect. In an effort to address these missed opportunities, what is advocated is an approach promoting Instrumentality in medical education, namely: facilitating trust in superiors, enabling greater resident control, and utilizing formative assessments in metric-based paradigms.

Valence

The final Expectancy Theory component for review is Valence. Valence is defined as the value an individual places on a reward.⁸ Expectancy Theory denotes Valence on a numeric spectrum: +1 reflecting a desired reward; 0 indifference; and −1 a non-desired reward.¹⁸ As described by Expectancy Theory, the closer the perceived Valence is to +1, the greater the individual’s motivation in achieving it.¹⁸

The following is a workplace example: an employer makes it known that increased compensation and workplace recognition will be given for work assignments completed earlier than stated deadlines. Expectancy Theory holds that valuations of this reward will be assessed differently by different employees, with an aspiring manager assigning this reward a Valence of +1 (an opportunity for increased salary and workplace advancement), whereas an employee without managerial aspirations assigns this a Valence of 0 or −1 (ie, not motivated to spend any extra time or effort completing the task).¹ Thus, as described by Expectancy Theory, maximizing an individual’s workplace motivation involves an understanding of the individualized views of rewards. Simply stated, the type of reward will impact a given individual’s motivation in achieving it. Expectancy Theory and Valence thus denote the concept that increasing an individual’s motivation in a task can be achieved merely by varying the reward, without changing the work.

Importantly, Valence refers to valuations not only of “concrete rewards” (eg, salary), but also of more abstract rewards such as workplace satisfaction, attitudes, and morale.¹⁸

Valence in application to residency training

As noted for Expectancy and Instrumentality, Valence also appears relevant to resident training. Residency training offers myriads of opportunities to assign rewards in response to performance efforts. For example, using paradigms described above, resident preoperative preparation for surgery can be rewarded by greater participation in the case, while prodigious patient care by ICU residents can be rewarded with greater autonomy. Both of these examples highlight opportunities in medical education to offer individually valued rewards, ie, positive Valence, in response to resident efforts. Enhanced motivation in training are proposed to follow.

Furthering the applicability of Valence in resident education is the opportunity to solicit individual resident input enunciating rewards they feel are of high value. For example, disparate residents might claim more independence in patient care; greater exposure to surgical skill training; or change in rotation schedule as of high Valence. Intuitively, soliciting such individualized Valence-related resident input could be of exceedingly high potential to increase resident motivation.

In sum regarding Valence and Expectancy Theory, understanding the varied valuations placed by individuals on rewards provides further insight into understanding resident motivation in training. Applying Expectancy Theory to residency education avails new opportunities establishing highly valued rewards. Such a methodology should afford enhanced resident motivation.

Having explicated Expectancy Theory and its components Expectancy, Instrumentality, and Valence in relation to residency training, an additional point in application is noteworthy. Namely, Expectancy Theory is predictably dynamic. For example, each component is likely to vary across postgraduate year (interns to chief resident) as well as each individual resident. Hence Expectancy Theory allows for great resident variability. One can therefore envision a scenario whereby a residency program director might distribute for their teaching faculty suggestions of Expectancy, Instrumentality, and Valence specific to each resident level, and even as related to each individual resident (Table 1).

Table 1 Expectancy, Instrumentality, and Valence applied to residency training Abbreviations: FLS, Fundamentals in Laparoscopic Surgery; ATOM, Advanced Trauma Operative Management; SICU, surgical intensive care unit; SCORE, Surgical Council on Resident Education.

The formative approach described in applying Expectancy Theory to residency training may appear at outset to be time-consuming and/or beyond available timeframes to be effective. These might then be intrinsic barriers in its application. Yet we propose that the myriad of continuous daily contacts between residents and more senior staff (eg, morning report, rounds, daily telecommunications, extended time together during after hours assignments (on-call), shared procedures, etc) provide innumerable and ample opportunity for substantive formative feedback, without necessitating designated and/or lengthy evaluations. In fact we propose that the structure of residency training, with its myriads of continuous interactions over long work hours, is ideal for the formative process proposed by Expectancy Theory. (The formative-type intervention study of the MSU GOAL noted above will further clarify this hypothesis).³¹

Of note, that a dynamic and individualized educational method is particularly relevant to contemporary residency training is depicted in a British Medical Journal article, the “ABC of learning and teaching in medicine. Curriculum design.” Stated is that residency teaching “[…] must be responsive to changing values and expectations in education if it is to remain useful”.³³ Seemingly, Expectancy Theory offers such a new perspective by which to view changing values and expectations in residency teaching.

Conclusion

Expectancy Theory is a well-known theory in work-motivation literature. It elucidates an understanding of the motivations behind an individual’s work-related decision making. Expectancy Theory involves the components Expectancy, Instrumentality, and Valence, forming a monotonic function summing in a Motivational Force.⁸

In aiming to meet advancing goals in contemporary residency education, we propose the application of Expectancy Theory to residency training. Innovative perspectives for gaining insight into resident motivation are afforded. What follows are opportunities to promote the development of programmatic methods enhancing resident motivation in education.

Acknowledgment

The authors wish to acknowledge the expertise of Lisa M Eckenrode regarding the table shown in this manuscript.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system – rationale and benefits. N Engl J Med. 2012;366(11):1051–1056.
2.	[No authors listed]. Enhancing standards for excellence in internal medicine training. Federated Council for Internal Medicine. Ann Intern Med. 1987;107:775–778.
3.	Accreditation Council for Graduate Medical Education [homepage on the Internet]. Chicago, IL: Accreditation Council for Graduate Medical Education. Available from: http://acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/ab_ACGMEglossary.pdf. Accessed February 8, 2012.
4.	Debas HT. Surgery: a noble profession in a changing world. Ann Surg. 2002;236:263–269.
5.	Debas HT, Bass BL, Brennan MF, et al. American Surgical Association Blue Ribbon Committee. American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004. Ann Surg. 2005; 241:1–8.
6.	Education [webpage on the Internet]. Chicago, IL: American College of Surgeons. Available from: http://www.facs.org/education. Accessed.
7.	The American Board of Surgery [homepage on the Internet]. Philadelphia, PA: The American Board of Surgery. Available from: http://www.absurgery.org. Accessed.
8.	Wigfield A, Eccles JS. Expectancy-Value Theory of Achievement Motivation. Contemp Educ Psychol. 2000;25:68–81.
9.	Pintrich PR, Schunk DH. Motivation in Education: Theory, Research, and Applications. Englewood Cliffs, NJ: Merrill-Prentice Hall; 1996.
10.	Eccles JS, Adler TF, Futterman R, et al. Expectancies, values, and academic behaviors. In: Spence JT, editor. Achievement and Achievement Motivation. San Francisco, CA: WH Freeman; 1983:75–146.
11.	Wigfield A. Expectancy-value theory of achievement motivation: a developmental perspective. Educ Psychol Rev. 1994;6:49–78.
12.	Wigfield A, Eccles J. The development of achievement task values: a theoretical analysis. Dev Rev. 1992;12:265–310.
13.	Bandura A. Self-Efficacy: The Exercise of Control. Worth Publishers. Duffield UK; 1997.
14.	Covington MV. Making the Grade: A Self-Worth Perspective on Motivation and School Reform. New York: Cambridge University Press; 1992.
15.	Weiner B. An attributional theory of achievement motivation and emotion. Psychol Rev. 1985;92:548–573.
16.	Deci EL, Ryan RM. Intrinsic Motivation and Self-Determination in Human Behavior. New York: Plenum Press; 1985.
17.	Deci EL, Vallerand RJ, Pelletier LG, Ryan RM. Motivation and education: the self-determination perspective. Educ Psychol. 1991;26:325–346.
18.	Vroom VH. Work and Motivation. New York: Wiley; 1964.
19.	Sullivan MC, Yeo H, Roman SA, et al. Surgical residency and attrition: defining the individual and programmatic factors predictive of trainee losses. J Am Coll Surg. 2013;216:461–471.
20.	Fundamentals of Laparoscopic Surgery [homepage on the Internet]. Los Angeles, CA: FLS. Available from: http://www.flsprogram.org/index/fls-program-description/. Accessed May 3, 2012.
21.	Swanstrom LL, Fried GM, Hoffman KI, Soper NJ. Beta test results of a new system assessing competence in laparoscopic surgery. J Am Coll Surg. 2006;202(1):62–69.
22.	Korndorffer JR Jr, Dunne JB, Sierra R, Stefanidis D, Touchard CL, Scott DJ. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg. 2005; 201:23–29.
23.	Stefanidis D, Sierra R, Korndorffer JR Jr, et al. Intensive continuing medical education course training on simulators results in proficiency for laparoscopic suturing. Am J Surg. 2006;191:23–27.
24.	McCluney AL, Vassiliou MC, Kaneva PA, et al. FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc. 2007; 21(11):1991–1995.
25.	Fried GM, Feldman LS, Vassiliou MC, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg. 2004;240:518–528.
26.	Jacobs LM, Gros RI, Luk SS. Advanced Trauma Operative Management: Surgical Strategies for Penetrating Trauma. Woodbury, CT: Cine-Med, Inc.; 2004.
27.	Jacobs LM, Burns KJ, Kaban JM, et al. Development and evaluation of the advanced trauma operative management course. J Trauma. 2003;55:471–479.
28.	Jacobs LM, Burns KJ, Luk SS, Marshall WT 3rd. Follow-up survey of participants attending the Advanced Trauma Operative Management (ATOM) Course. J Trauma. 2005;58:1140–1143.
29.	Ali J, Ahmed N, Jacobs LM, Luk SS. The Advanced Trauma Operative Management course in a Canadian residency program. Can J Surg. 2008;51:185–189.
30.	Jacobs L, Burns K, Luk S, Hull S. Advanced trauma operative management course: participant survey. World J Surg. 2010;34:164–168.
31.	msusurgerygoal.com [homepage on the Internet]. Lansing, MI; Michigan State University – Department of Surgery; 2013. Available from: https://www.msusurgerygoal.com/login.php. Accessed December 20, 2014.
32.	Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education. J Am Coll Surg. 2007;204:341–346.
33.	Prideaux D. ABC of learning and teaching in medicine. Curriculum design. BMJ. 2003;326:268–270.