Procedural skills practice and training needs of doctors, nurses, midwives and paramedics in rural Victoria

Introduction

The use of procedural skills is a significant component of a clinician’s overall professional practice. All health disciplines are trained to carry out a range of simple to complex procedural skills. Further development, refinement, and mastery of some or all of these skills can occur in many circumstances through repeated clinical experience.¹ In some settings, however, opportunities for maintenance of procedural skills may be less than optimal. This is especially true in rural and remote environments, where clinical opportunities to undertake particular skills may be infrequent, and access to continuing education may be limited.² This can lead to reduction in levels of confidence and competence, leaving the clinician in a lessened state of preparedness, capability, and capacity to carry out a procedural skill when required to do so. Consequently, there are risk and safety ramifications for the clinician, for health services, and for the community, particularly given the increasing focus on quality and safety among professional and regulatory bodies.^3–8

Previous research in this area has produced only limited evidence regarding the nature of training and retraining required for maintenance of clinicians’ confidence and competence in procedural skills.^9–14 Training and retraining become a challenge for any health care service, which requires a workforce to deliver safe and competent care through procedural skills and other discipline-specific practices. Rural and remote health care services find this challenge particularly demanding due to a number of reasons such as retention and recruitment factors, time and distance concerns, exposure to appropriate patients and clinical experiences, and appropriate education resources.^2,13,15–20 In addition, clinicians vary in their perceptions and attitudes regarding the frequency of retraining and rehearsal of procedural skills.²¹

To address this confidence–competence phenomenon in such settings, one strategy is to carry out a procedural skills survey. This activity helps ascertain the current workforce scope of procedural skills practice, helps identify where there may be skill deficits within that scope, and provides relevant information to develop and deliver focused education and training to mitigate this gap.²²

The aim of this study was to scope the procedural skills practice and training needs of doctors, nurses, midwives, and paramedics in the Gippsland region of Victoria, Australia. We were unable to identify any comprehensive data relevant to clinicians’ practices in Gippsland. With the advent of significant investment in simulation resources in Gippsland, through the establishment of simulated learning environments (SLEs), including simulation mannequins, equipment, and project officers, with funding from Health Workforce Australia, we wanted to better understand current practice and perceived needs for procedural skills training.^23,24 This data may enable the development of training opportunities that are relevant and timely. It is anticipated that this may contribute to improved quality and safety of procedural skills. Offering high-quality and relevant training may also improve recruitment and retention of clinicians.

The research questions were the following:

• What is the current scope of procedural skills practice of nurses, midwives, doctors, and paramedics in Gippsland?
• What are their perceived training needs in procedural skills?
• What is the role of simulation in procedural skills training and maintenance?

Methods

The research questions were best suited to questionnaire-based data collection. This was the most efficient method of contacting the target group dispersed over a large geographical area. Individual or focus group interviews would have been expensive and time-consuming and were beyond the scope of the project. The questionnaire contained both quantitative and qualitative questions, and was designed to provide adequate information to answer the research questions.

Online questionnaires

We developed an online questionnaire for each discipline. The questionnaires consisted of two parts – demographic and professional characteristics, and experience of procedural skills. The first section covered demographic information. This included questions identifying respondents’ age, sex, profession, current employment position, main area of work, their highest qualification and how many years since their first professional qualification was obtained, their country of training for their first professional qualification, and whether they worked full time or part time, whether they were on a “permanent” contract or a fixed-term contract, or worked as a casual employee. The last question focused on the type of clinical environment in which they worked.

The second section explored respondents’ experiences of procedural skills and their perceived training needs. We were unable to find comprehensive lists of procedural skills for disciplines. Questionnaire development took several months. The process for each discipline was similar and is summarized in Figure 1. First, we identified discipline-specific curriculum or professional practice documents. Second, up to four active clinicians in each discipline reviewed the lists, adjusting to reflect their experiences. Third, questions were developed that explored an individual clinician’s experience of each procedural skill. This questionnaire was then piloted with three clinicians for each discipline. The questions and lists of procedural skills were adjusted resulting in the final list containing 55 skills for doctors, 25 for nurses, 56 for midwives, and 57 for paramedics. There is considerable overlap in the procedural skills identified by each discipline group, as well as discipline-specific skills. The number of procedural skills identified for nurses is approximately half that of the other disciplines; this reflects both the scope of procedural skills practice and the process of identification of appropriate procedural skills for the purposes of the study.

Figure 1 Development process for questionnaire.

For each procedural skill, there were five questions to be answered. The first question ascertained when the respondent last performed the procedure; the second question asked how confident they felt when performing the skill (a Likert-type scale from not at all confident [1] to completely confident [5]). Third, respondents were asked how they maintained competence by choosing from a list of options, including the use of simulation. This was followed with questions that ascertained if the respondent had ever learned procedural skills using simulation, which included task trainers, mannequins, or simulated patients. Finally, participants were asked that if they had access to simulation now, how often they thought they would need to practice these procedural skills to maintain competence. Once again, a range of timelines were offered. The questionnaire was conducted on SurveyMonkey © 2011. Supplementary materials contain the questionnaires for each discipline.

Sampling and data collection

We sought to identify all clinicians (doctors, nurses, midwives, paramedics) working in Gippsland. To access each discipline, we needed to use different approaches. It was not possible to get a precise number of clinicians across Gippsland. At best, we could make estimates, although it was easier to access some populations than others. Members of the research team who lived and worked in Gippsland provided local knowledge that facilitated access to practice groups. Online questionnaires were distributed between November 2011 and April 2012. Follow-up was conducted on three occasions.

Invitations to clinicians

For general practices, we worked through administrative units and training provider organizations to access participants. We also sought to access hospital-based doctors, nurses, and midwives employed in the 12 acute and community health services in Gippsland. All potential clinicians were accessed via their payroll systems. As the Gippsland region has a number of bush nurse centers, it was deemed important to invite these nurses. There are a small number of independent practice midwives in the region, so these clinicians were also invited to undertake the survey. Ambulance Victoria (AV), the administrative organization for paramedics, recommended that paramedics be contacted only via the Gippsland “staff portal” of their internal intranet, and they also provided facsimile numbers of local branches which allowed a paper version of the questionnaire invitation to be sent.

Analysis

Data from this research were analyzed using descriptive statistics (frequencies, means, medians, range, and variance). Exploratory results for each procedural skill (frequency of performing, confidence to perform, competence maintenance, and simulation) have been determined separately by clinician type. Statistical associations between frequency of performing and i) confidence to perform, and ii) desired skill maintenance through simulation, by individual procedural skills were measured using Spearman’s rank-order (nonparametric) correlations. Free text comments were analyzed thematically and summarized.

Ethics

Primary ethics approval was obtained from Monash University Human Research Ethics Committee (CF11/3013 – 2011001701). AV and each regional hospital ethics committee also approved the study. The multiple ethics approval processes took significantly longer than anticipated, and the requirements of each group to access participants placed significant barriers on response rates.

Results

Response rates

Valid responses, of both demographics and procedural skills, were received from 58 doctors, 94 nurses, 46 midwives, and 30 paramedics. A summary of the respondents’ demographics is contained in Table 1. The precise number of each professional group in Gippsland is unknown.

Table 1 Demographics of respondents by clinician type

Most responding doctors were under 45 years of age (73%) and worked in general practice (65%), and only a small majority were Australian-trained (61%). In contrast, almost all nurses were female, Australian-trained (93%), over 45 (61%), and worked part time (76%) in public hospitals (66%) and in a wide range of positions. All midwives were female, located in a public hospital (100%), and mostly worked part time (83%), while all paramedics worked full time in either mobile intensive care (30%) or advanced life support (ALS) (50%) positions (Table 2). Without having comparative population-level statistics, we are unable to assess the representativeness (or bias) inherent within each cohort.

Table 2 Clinical characteristics of respondents by clinician type Abbreviations: ALS, advanced life support; MICA, mobile intensive care ambulance; GP, general practice.

Most clinicians reported having previous simulation experience, ranging from midwives (83%) to nurses (94%) (Table 3). All four groups reported this was most likely from mannequins (all >80%), with most midwives and paramedics also reporting experience in training with simulated patients. Experience using task trainers was reported by about 50% of respondents across the four clinician groups.

Table 3 Simulation experience of respondents by clinician type

Frequency of procedural skills use

One hundred and ninety-three different procedural skills across the four groups were surveyed. Table 4 shows that a majority of skills were not performed within the last week. Doctors clearly performed the least number of skills on a regular basis. For almost 80% of the listed skills for doctors, less than 20% of doctors had recently performed that skill. In contrast, more than half of the midwives and paramedics reported that they had performed around 40% of the listed skills in the last week.

Table 4 Frequency of recent use of procedural skills by clinician type Note: *Definition: >50% of respondents reported using skill either yesterday or last week.

Procedural skills practice

Table 5 includes the doctors’ results of when each procedural skill was last performed together with the clinician’s confidence to perform that skill, with correlations provided to measure their association. For efficiency, Tables 6 and 7 include results for five randomly selected skills from each clinician group. The full set of results can be downloaded from the journal (Supplementary materials). Generally, procedural skills for which clinicians expressed low confidence were also those that were rarely or even never performed (eg, Bier’s Block for doctors). In contrast, high confidence was often associated with recent skill usage (eg, noninvasive blood pressure measurement for midwives). However, it could also be associated with moderate-to-low-usage skills such as oropharyngeal suctioning for paramedics. This latter outcome suggests that there are some “core” or “fundamental” skills for which clinicians remain highly confident despite only moderate use. Among nurses who completed the questionnaire, clinical procedures which are not mandated as annual competencies, or not often practiced, were associated with low confidence. However, emergency skills such as basic life support or ALS and cervical spine immobilization that were infrequently used in practice but which received high-confidence ratings are examples of skills for which many nurses are required to achieve competency on an annual basis.

Table 5 Doctors’ practices of procedural skills and confidence of use Abbreviations: CXR, chest X-ray; ECG, electrocardiograph; AED, automated external defibrillator; CPAP, continuous positive airway pressure; BiPAP, bilevel positive airway pressure; DC, direct current.

Table 6 Examples of recency of performance for five randomly selected skills and confidence of use (confidence rated on a five-point scale) Abbreviations: ECG, electrocardiograph; BP, blood pressure; MET, medical emergency team.

Table 7 Respondents’ preferred method for maintaining procedural skills and frequency of simulation Abbreviations: ECG, electrocardiograph; BP, blood pressure.

Table 7 summarizes how competence in procedural skills is currently maintained by clinicians and correlates frequency of usage of that skill with use of simulation to practice the skill. Those skills of highest usage were clearly also those most likely to be maintained through repeated practice or direct observation. A few skills were currently practiced at a moderate level through simulation (eg, endotracheal tube intubation (ETT) for doctors and pediatric resuscitation for midwives).

Nearly half of all doctors maintained competence in airway skills such as laryngeal mask airway (LMA) and ETT insertion through practice in simulation, and two-thirds of doctors thought that these skills should be practiced in simulation at least every year. This compares with paramedics, 88% of whom thought that LMA should be practiced by simulation at least every year (correlating with 79% reporting high confidence with this skill), although only 44% thought that ETT insertion should be practiced by simulation this frequently, with 63% of paramedics reporting that they maintained competence in ETT insertion by direct clinical observation or peer advice, despite less than half of them reporting high confidence with this skill.

The expressed frequency of use of simulation (if available) generally did not change greatly with specific procedural skills. In addition, the association between frequency of usage and desired simulation training frequency was generally quite low or even negative with most skills. The exceptions (ie, skills with a high correlation between frequency of use and desire for frequent simulation training) were the following: cardiac rhythm interpretation, ALS, maintaining a patent airway, and splinting/immobilization among paramedics; management of postpartum hemorrhage among midwives; and no such correlations among the doctor and nurse groups.

It is also instructive to identify the most commonly performed procedures in each professional group, and the attitudes to maintaining competence in these procedures. Among doctors, the skills that 80% or more of respondents reporting using within the past 6 months were oxygen therapy, chest X-ray (CXR) interpretation, electrocardiograph (ECG) interpretation, plastering, and suturing. Simulation training in these skills any more often than annually did not rate highly, with the majority of doctors reporting that they maintained clinical competence through repeated clinical practice or clinical observation. Within the nurse group, the skills used by more than 70% of respondents were vital signs measurement, basic airway management, urinary catheter management, administration of medication, and wound/stomal care. Attitudes to maintenance of competence in these five skills were similar to the attitudes reported for the five main skills among doctors.

Among midwives, approximately half of the listed skills were reported by more than 50% of the cohort as being used in the past week. The skills most commonly used were vital signs measurement, pulse oximetry, intravenous fluid management, postnatal care (including breastfeeding), and medication administration (all routes). Again, repeated clinical practice was the most highly rated means of maintaining these skills.

For paramedics, there were seven skills used by 100% of respondents in the past week. These skills were primary survey, vital signs measurement, chest auscultation, pulse oximetry, neurological assessment, cardiac rhythm interpretation, and drug administration. Interestingly, despite reporting high confidence with these skills, more than 50% of respondents thought that five of these skills should be practiced by simulation at least every 6 months.

Among the doctor respondents, the majority of skills included in the questionnaire were considered to be relevant to their practice. Peritoneal dialysis was the skill attracting the highest percentage of doctor respondents (24%) who regarded this skill as not relevant to their practice. Not surprisingly, 86% of respondents reported low confidence with this procedure.

Some of the obstetric procedures also attracted a similar response (around 22%) from doctors who regarded these skills as not relevant to their practice. We are unable to extract from the data how many of these doctors are in active obstetric practice, but it is reasonable to assume a correlation between these factors. In addition, there is evidence of a strong correlation between frequency of performance of some obstetric skills and confidence levels. For example, management of both shoulder dystocia and cord prolapse occurs relatively infrequently in clinical practice among respondents, and attracts a low confidence score in more than 60% of respondents.

Conversely, ECG interpretation is reported by 75% of respondents as having been carried out in the preceding week; no respondents reported this skill as irrelevant to their practice, 71% reported maintenance of this skill in repeated clinical practice, and only 45% thought this should be practiced in a simulated environment at least every year.

Similarly, CXR interpretation enjoys similar standing among doctor respondents. Ninety-three percent of respondents had carried this out within the preceding 6 months, and 89% felt highly confident in their abilities. This corresponds to 65% reporting clinical practice as the best way to maintain this skill, and only 6% feeling they needed to rehearse this in a simulated environment.

For a skill less commonly used such as intraosseous needle insertion (only 2% had carried this out within the past 6 months), only 8% of respondents thought this was not relevant to their practice, 52% reported low confidence with the procedure, 43% thought this should be practiced by simulation, and 55% thought this should occur at least every year.

Discussion

Interpretation of principal findings

The findings on the current scope of procedural skills practice of doctors, nurses, midwives, and paramedics in Gippsland are consistent with expectations. The relevance of the skills chosen for the study is discussed below.

There are some interesting perspectives to be drawn from the findings on the perceived training needs in procedural skills among the professional groups. The greatest correlation between the listed skills and relevance of these skills to their practice was among the paramedics and midwives. This could relate to the narrower scope of clinical practice in paramedical and midwifery practice compared to medical and nursing practice, or could represent a more rigorous process of identification of core procedural skills in the development of the questionnaire. It is nevertheless true that the skills listed for paramedics and midwives are mostly related to the specific requirements of emergency and maternity care, respectively; those listed for doctors and nurses cover a broader range of clinical environments and consequently a broader scope of clinical practice. Both the process by which the questionnaires were developed and the questionnaires themselves are an important outcome of the study.

The professional group that reported the strongest correlation between confidence in frequently used skills and a desire to practice those skills relatively frequently by simulation was the paramedics. This could reflect the often high-stakes nature of these skills, where satisfactory employment of these skills could mean the difference between life and death of the patient.

Attitudes to maintenance of skills

This questionnaire provides extensive information about clinicians’ attitudes to the frequency of “upskilling” required to maintain competence in a range of procedural skills. This has been examined in significant detail in a meta-analysis of the literature.²⁵ As part of their study, on the maintenance of knowledge, attitudes, skills, and practice behavior outcomes persisting over time, Marinopoulos et al concluded, “little can be said about the effectiveness of CME for psychomotor skills given the paucity of data in this area”.²⁵ In addition,

given the limited number of studies … it is difficult to draw conclusions about the education techniques that have the greatest short- and long-term effects on skill … Most of the studies that met their skills objectives had multiple exposures to the CME activity as did most of the studies that evaluated the long-term effects on skills.

This evidence is supported by a more recent systematic review of retention of ALS knowledge and skills in health care providers.²⁶ This review concluded that

evidence suggests that ALS knowledge and skills decay by 6 months to 1 year after training and that skills decay faster than knowledge

but the review also acknowledged the large well-designed studies in this area.²⁶

It is clear therefore that the current literature is inadequate with respect to informing the most appropriate educational modalities for learning and maintaining procedural skills, as well as the ideal frequency for revision of these skills. It seems likely that cognitive skills can be retained for some time, and that multiple exposures to the learning activity are likely to embed these cognitive skills over the longer term. There are insufficient data on retention of psychomotor skills.

Our study did not ask participants to distinguish between the cognitive and psychomotor aspects of procedural skills, but has principally examined the attitudes of a range of clinicians toward how, and how often, they should rehearse the skills listed. This provides valuable evidence that informs the design of programs for continued procedural skills education, including both cognitive and psychomotor aspects of the skills, and maximizes the opportunity for engagement in such programs by local clinicians.

We have found that generally most clinicians retain a high level of confidence in skills that they use frequently, or skills in which they are required by their profession to maintain competence regularly. Apart from paramedics as noted above, the majority of clinicians did not feel a need to practice these skills by simulation any more often than annually, if at all.

Conversely, they identify lower levels of confidence in skills they use rarely or infrequently, and believe they need to rehearse these skills by a variety of means, including by simulation at least 6–12 monthly. For many of these rarely used skills, practice by simulation also correlated highly with clinical practice or clinical observation as a means of retaining competency in the skill. There were exceptions to this finding, for example, surgical airway insertion among doctors, but generally the findings demonstrate that clinical practice and clinical observation can be backed up by training via simulation to the satisfaction of most respondents. This varies from skill to skill depending on its complexity and the extent to which the clinical environment provides realistic opportunities for practicing the skill.

Use of simulation

Nestel et al²⁷ found that trainees and instructors express high levels of satisfaction with simulation as an educational method. Simulation usually results in improved knowledge and skills, but most studies have focused on short-term gains in knowledge and skills, with outcomes usually tested in simulation rather than in clinical practice. In the few studies that have evaluated the impact of learning on clinical practice, the evidence is positive.²⁸ “Simulation not only supports learning but retention and revalidation of procedural skills. We need to understand more about the complex relationships of timing and duration of procedural skills training, frequency of use of the skill, initial competence and skill decay”.²⁷ Our study supports these findings, to the extent to which clinicians regard simulation as a useful method of retention of competence in a range of skills relevant to clinical practice, particularly those skills that are used rarely or infrequently, or “high-stakes” skills on which the survival of the patient may depend. This will assist in the design of specific skills training programs in SLEs and “skills laboratories”.

Strengths and limitations of the study

Although the implementation of the questionnaire posed many challenges, the process for developing the questionnaire was robust. Some respondents added additional procedures, but these were almost all singular suggestions. The response rate was disappointing, and reflects several factors involved in the design and conduct of the study, including some barriers encountered in distributing the questionnaire to all groups identified in the study. This reflected some variation in engagement with organizations at critical points of access to the workforce for distribution of the questionnaires. This involved the need for repeated approaches to key individuals, and this occupied a considerable amount of time for the researchers in the early part of the study. The requirements of privacy legislation prevented any alternative means of accessing workforce databases.

The implications for clinical practice are far reaching. From each discipline’s perspective, there are a range of procedural skills that need to be maintained at a level expected by the state, the community, and the professions to ensure safe and competent practice continues. Evidence that there are real or potential gaps in procedural skills practice provides an opportunity to target and develop training programs to address and reduce that gap. A further implication is an understanding of the reasons for gaps in procedural skills in a region such as Gippsland and similar rural and remote areas. The literature and data indicate a range of factors that impact on the sustainability of procedural skills. This then provides an opportunity to look at different and innovative approaches to address some of those issues, such as the provision of timely, focused, and relevant education and training.

There is a developing evidence base around the use of simulation education to help maintain competence and confidence in a wide range of procedural skills for various disciplines.^29–34 The use of simulation to encourage an interdisciplinary and interprofessional approach to supporting existing and future workforce personnel is a further development.³⁵ Such activities, along with the Health Workforce Australia national agenda in supporting the development of SLEs through investment in infrastructure, programs, and human resources to increase the use of simulation, may be a strong catalyst for positively impacting on procedural and other workforce skills.³⁶

Conclusion

This study achieved the research aims of identifying the scope of practice of procedural skills of a sample of clinicians in the Gippsland region, as well as identifying the training and upskilling needs and attitudes of these clinicians through analysis of complex interrelated data from each discipline group. The results of this study provide the basis for bringing together clinical experts, health executives, policy makers, educators, and researchers to collaborate in the design, delivery, and evaluation of focused education interventions. It also allows for innovation in how and where that education may occur and in formats that engage the region’s workforce more effectively.

Acknowledgments

The authors wish to thank the following for their support in this project: Marion Bowron (Gippsland Region Nurse and Midwife Education Coordinator), Melanie Regan, Letitia Clark (Southern General Practice Training), and Doug Gallagher (Ambulance Victoria). Additionally, we acknowledge the Australian Government through the Australian General Practice Training Program for funding the project.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Kneebone R, Nestel D. Learning and teaching clinical procedures. In: Dornan SE, editor. Medical Education: Theory and Practice. London: Elsevier; 2010:172–192.
2.	Australian College of Rural and Remote Medicine (ACRRM). Barriers to the Maintenance of Procedural Skills in Rural and Remote Medicine. In: ACRRM Research Projects; ACRRM, Kelvin Grove; 2002:1–42.
3.	Health Workforce Australia (HWA). Use of Simulated Learning Environments (SLE) in Professional Entry Level Curricula of Selected Professions in Australia; 2010. Available from: http://www.hwa.gov.au/sites/uploads/simulated-learning-environments-2010-12.pdf.
4.	World Alliance for Patient Safety. WHO Patient Safety Curriculum Guide for Medical Schools; 2009 [cited July 24, 2011]. Available from: http://www.who.int/patientsafety/information_centre/documents/who_ps_curriculum_summary.pdf.
5.	Cooper JB, Gaba DM, Liang B, Woods D, Blum LN. The National Patient Safety Foundation agenda for research and development in patient safety. MedGenMed. 2000;2(3):E38.
6.	Kohn L, Corrigan J, Donaldson M. editors. To Err is Human: Building a Safer Health System. I.o.M. Committee on Quality of Health Care in America, editors; National Academy Press: Washington DC; 2000.
7.	Health Workforce Australia. Health Workforce Australia 2012: Health Workforce 2025 – Doctors, Nurses and Midwives – Volume 1; 2012. Available from: http://www.hwa.gov.au/health-workforce-2025.
8.	Health Workforce Australia. Health Workforce Australia 2012: Health Workforce 2025 – Doctors, Nurses and Midwives – Volume 2; 2012. Available from: https://http://www.hwa.gov.au/sites/uploads/health-workforce-2025-volume-2.pdf.
9.	Curry L. Effects of training in cardiopulmonary resuscitation on competence and patient outcome. CMAJ. 1987;137:491–496.
10.	Bishop M. Recertification of respiratory therapists’ intubation skills one year after initial training: an analysis of skill retention and retraining. Respir Care. 2001;46:234–237.
11.	Vertongen V. Skills retention for insertion of the Combitube and laryngeal mask airway. Emerg Med. 2003;15(5–6):459–464.
12.	Bould M, Crabtree N, Naik V. Assessment of procedural skills in anaesthesia. Br J Anaesth. 2009;103(4):472–483.
13.	Glazebrook R, Harrison S. Obstacles to maintenance of advanced procedural skills for rural and remote medical practitioners in Australia. Rural Remote Health. 2006;6(4):502.
14.	Hammond FH. Advanced life support: retention of registered nurses’ knowledge 18 months after initial training. Aust Crit Care. 2000;13:99–104.
15.	Hays R, Evans R, Veitch C. The quality of procedural rural medical practice in Australia. Rural Remote Health. 2005;5(4):474.
16.	Hundley VA, Tucker JS, van Teijlingen E, et al. Midwives’ competence: is it affected by working in a rural location? Rural Remote Health. 2007;7(3):764.
17.	McAuliffe T, Barnett F. Factors influencing occupational therapy students’ perceptions of rural and remote practice. Rural Remote Health. 2009;9(1):1078.
18.	Smith D. Barriers facing junior doctors in rural practice. Rural Remote Health. 2005;5(4):348.
19.	Montour A, Baumann A, Blythe J, Hunsberger M. The changing nature of nursing work in rural and small community hospitals. Rural Remote Health. 2009;9(1):1089.
20.	Veitch C, Dew A, Bulkeley K, et al. Issues affecting therapist workforce and service delivery in the disability sector in rural and remote New South Wales, Australia: perspectives of policy-makers, managers and senior therapists. Rural Remote Health. 2012;12(2):1903.
21.	Taylor D. Procedural skills quality assurance among Australian College for Emergency Medicine fellows and trainees. Emerg Med Australas. 2006;18:268–275.
22.	Laird C, Ker J, Stevenson J. A Training Needs Assessment of Remote and Rural Healthcare Practitioners to Inform the Skills Education Delivered through the Mobile Clinical Skills Unit. Clinical Skills Managed Educational Network. 2009;2009:1–34.
23.	Health Workforce Australia. Simulated Learning Environments; [cited July 17, 2012]; Available from: http://www.hwa.gov.au/work-programs/clinical-training-reform/simulated-learning-environments-sles.
24.	Nestel D, Watson MO, Bearman ML, Morrison T, Pritchard SA, Andreatta PB. Strategic approaches to simulation-based education: a case study from Australia. J Health Spec. 2013;1(1):4–12.
25.	Marinopoulos S, Dorman T, Ratanawongsa N, et al. Effectiveness of continuing medical education. Evid Rep Technol Assess (Full Rep). 2007;149:1–69.
26.	Yang CW, Yen ZS, McGowan JE, et al. A systematic review of retention of adult advanced life support knowledge and skills in healthcare providers. Resuscitation. 2012;83:1055–1060.
27.	Nestel D, Groom J, Eikeland-Husebø S, O’Donnell JM. Simulation for learning and teaching procedural skills: the state of the science. Simul Healthc. 2011;6(Suppl):S10–S13.
28.	Ker J, Bradley D. Simulation in medical education. In: Swanick T, editor. Understanding Medical Education: Evidence. Theory and Practice. Oxford: Wiley-Blackwell; 2010:164–180.
29.	Nestel D, Kneebone R. Authentic patient perspectives in simulations for procedural and surgical skills. Acad Med. 2010;85(5):889–893.
30.	McGaghie WC, Siddall VJ, Mazmanian PE, Myers J, American College of Chest Physicians Health and Science Policy Committee. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009;135(3 Suppl):62S–68S.
31.	Kneebone R. Evaluating clinical simulations for learning procedural skills: a theory-based approach. Acad Med. 2005;80(6):549–553.
32.	Kneebone RL, Nestel D, Vincent C, Darzi A. Complexity, risk and simulation in learning procedural skills. Med Educ. 2007;41(8):808–814.
33.	Okuda Y, Bryson EO, DeMaria S Jr, et al. The utility of simulation in medical education: what is the evidence? Mt Sinai J Med. 2009;76(4):330–343.
34.	Ypinazar VA, Margolis SA. Clinical simulators: applications and implications for rural medical education. Rural Remote Health. 2006; 6(2):527.
35.	Baker C. Simulation in interprofessional education for patient-centred collaborative care. J Adv Nurs. 2008;64(4):372–379.
36.	Health Workforce Australia (HWA). Clinical Training Reform; n.d. Available from: http://www.hwa.gov.au/work-programs/clinical-training-reform.