Simulation-Based Instruction in Continuing Education

KEY POINTS

• Simulation-based interventions, as part of a continuing professional education course (CPE), successfully led to the acquisition of knowledge and skills specific to a comprehensive cardiovascular assessment.

• In this study, high-fidelity simulation improved skill more than low-fidelity simulation because of the active nature of the learning experience.

• Opportunities to check maintenance of competence of clinical skills that are low frequency but high risk for patients can be created by implemented simulation into continuing professional education, which allows clinicians to repetitively practice and refine such skills while posing no risk to real patients.

INTRODUCTION

Continuing professional education (CPE) is a vehicle for promoting the development of clinical knowledge and skills that allow practitioners to stay abreast on the changing demands in their clinical practice.¹ Health care professionals, such as athletic trainers (ATs), must participate in CPE to maintain the knowledge and skills required to be a competent clinician. A common structure for CPE courses is a lecture presented by an expert to a large group of individuals.² This CPE delivery method has been identified as having little impact on professional practice³ because of the passive learning approach. Learning is more effectively enhanced through active participation,^4,5 and measuring learning gains rather than attendance and satisfaction⁶ can provide more insight into the effectiveness of the CPE course.

The literature on deliberate practice,^7–9 mastery learning,^10–12 and experiential learning theory^13–15 supports the notion that active participation enhances learning. To keep up with best practices in education, it may be beneficial to add active learning approaches to CPE courses. Active learning can be achieved through simulation,¹⁶ which has been documented as an effective technique to engage learners by providing hands-on experiences^15,17 for teaching trainees, especially medical and nursing students, relevant knowledge and clinical skills specific to cardiopulmonary resuscitation (CPR),^18,19 cardiorespiratory assessments,²⁰ pulmonary assessment,²¹ and surgical procedures.²² Because of the limited availability of patients with clinical abnormalities, simulation presents itself as a vehicle for students and professionals to practice, develop, and refine relevant clinical knowledge and skills.²² Little research has been done on the effectiveness of using simulation in CPE,²³ especially with ATs.

There are different simulation types that vary in authenticity, but provide the same focus on active learning. High-fidelity and low-fidelity simulation both require the learner to actively maneuver an experience, such as manipulating the environment. The higher the fidelity of a simulator, the more realistic the simulation.²⁴ A computer-enhanced mannequin is the most refined human patient simulator, and, because of its advanced technology, has the ability to replicate normal or abnormal physiological functions.^25,26 The human patient simulator Harvey (Laerdal Corp, Stavanger, Norway), is a tool that can be used for CPE to ensure the maintenance of clinical knowledge and skill.²⁵ A standardized patient (SP) can also be used for a high-fidelity simulation, as the SP is trained to portray a real medical situation.²⁷ Integrating SP encounters into ATs' professional education has been found valuable.²⁸ Low-fidelity simulators can include part-task trainers or computer programs; generally, the scenario is less realistic and such simulators are implemented to focus on one specific task or body part.²⁵ Regardless of the simulator's fidelity, simulation provides the learner the ability to actively practice and engage with the content to practice and reinforce knowledge and psychomotor skills.⁹

One area of knowledge and psychomotor skill that is a topic of prominence in athletic training is cardiac assessment.^29,30 Detecting cardiac disease is the focus of the preparticipation cardiovascular examination. Professional education requires students to be able to assess and interpret the findings of a cardiovascular examination.³¹ The National Athletic Trainers' Association (NATA) has published a position statement on the preparticipation physical examination³² and translational research of clinical practice in the form of building blocks specific to cardiac assessment.^29,30 Because professional education expects students to recognize cardiac abnormalities and make appropriate referrals to a cardiologist, ATs should be able to conduct a cardiovascular assessment that includes a physical examination²⁹ and basic cardiac auscultations³⁰ to recognize abnormal versus normal findings. Because of the relevance of the skill to ATs and the effectiveness of using simulation to teach cardiovascular screening,^33,34 we decided to reinforce ATs' knowledge and skill of a comprehensive cardiovascular assessment using simulation as part of a CPE course.

Although simulation is useful, scant literature exists investigating its effectiveness as part of CPE. Therefore, the purpose of this research was to compare the effects of 2 simulation-based instructional strategies (high- versus low-fidelity) on ATs' clinical knowledge and skill in performing a comprehensive cardiovascular assessment and use the results to provide direction for integrating simulation into CPE for ATs.

METHODS

Experimental Design

This research was conducted with the aim to compare the effects of 2 different simulation-based instructional strategies on ATs' clinical knowledge and skill in performing a comprehensive cardiovascular assessment. To achieve this goal, a randomized pretest-posttest group design was used to measure knowledge and skill acquisition at a CPE course before and after an intervention. The CPE course was 1 day and took place at a simulation center at a university after approval by the institutional review board. The 1-day course required the participants to complete the following (Figure 1): a knowledge and skill pretest, a 3-hour learning session, a 1-hour simulation intervention (high-fidelity or low-fidelity simulation), and a knowledge and skill posttest.

View Figure

• Download as Powerpoint
• Download Figure

Participants

Athletic trainers in south Florida were recruited to attend the CPE course. Participants had to meet the following inclusion criteria: (1) be an AT in good standing and certified with the Board of Certification, Inc; (2) be licensed within the state the course was offered; (3) be certified in CPR; and (4) be certified in the use of an automated external defibrillator. A formal recruitment e-mail with the continuing education unit (CEU) course flyer was sent to a variety of program directors at local institutions and ATs in south Florida to elicit participants for this study. All recipients were asked to forward the e-mail to their local colleagues who met the criteria. Anyone interested in attending the CEU course was asked to e-mail the organizers (researchers) of the event. Those participants who contacted the organizers were sent a consent form to review before attending the course and a designated arrival time to avoid wait time, as a pretest was a requirement for each participant. Additionally, within the e-mail, information was provided on the interactive nature of the course. All participants were asked to wear athletic clothing to permit active and interactive participation during the laboratory sessions.

Because participants registered for the course, we were able to assign them to an independent-variable simulation-type group before the course began. The assignment to a group occurred randomly: we placed all registered participants' names into an envelope, picked out one after the other, and placed them alternatingly into one of the 2 simulation type groups. The simulation type had 2 levels (high- and low-fidelity). A total of 30 participants registered, but only 22 attended, consented, and completed the course. Ten of the 22 participants were assigned the low-fidelity group and 12 of the 22 participants were assigned the high-fidelity group. Because all 22 participants who attended the course also consented and completed the course, there was a 100% participation rate. The participants' demographics are specified in Table 1. Upon arrival to the course, each participant checked in and was sent to the simulation laboratory to individually complete the 4 stations of the objective structured clinical examination (OSCE).

Table 1 Participant Demographic Information

View Fullscreen

Instruments

Two instruments were used to measure participants' knowledge and skill pre and post intervention: (1) a 31-item multiple-choice examination to assess participants' knowledge and (2) an OSCE to assess participants' cardiovascular assessment skills. Both instruments were developed and then reviewed by a panel of experts, including a cardiologist, nurse anesthetists, physical therapists, and ATs, for content and face validity. Revisions were made based on their feedback to ensure the relevance and clarity of each item.

The knowledge examination and OSCE were created to reflect the content of the educational learning session. The content was framed by the 7 learning objectives listed in Table 2.

Table 2 Objectives of Continuing Professional Education Course

View Fullscreen

Objective Structured Clinical Examination

The OSCE was used to assess the participants' skills. The OSCE aligned with objective 1 of the CPE course (Table 2); it was based on the NATA Building Blocks for Cardiac Assessment^29,30 and the NATA Position Statement on Preparticipation Physical Examinations, which include recommendations from the American Heart Association.³² Two components (4 stations) comprised the OSCE: an 8-item history-taking assessment and an 11-item clinical skill assessment. To assess the history-taking^29,32 and clinical skills components,^29,30,32 4 individual OSCE stations (Figure 2) were developed. The maximum score of the entire OSCE was 20. That score was composed of maximum scores earnable at each station (Figure 2). All stations were scored dichotomously (0 = incorrect, 1 = correct). A higher score indicated greater skill. Research indicates that the OSCE is valid and reliable in assessing the competence and clinical performance of psychomotor skills. Navas-Ferrer et al³⁵ analyzed the validity and reliability of the OSCE used in nineteen studies with nursing students and reported a common internal consistency score from moderate to high (0.51–0.94). The validity of the OSCE for a usual clinical assessment was reported as P = .536 (P < .01)³⁶ and r = 0.523 (P < .005).^35,37

View Figure

• Download as Powerpoint
• Download Figure

As a pretest and posttest, the OSCE was set up in a simulation laboratory. Each station was located in a separate room but in close proximity to the others. The participants' transitions and sequential flow from station to station were regulated by a facilitator located in the hallway outside the stations. This facilitator helped only on the day of the CPE. Once the participant entered a station, an evaluator read a prompt and assessed the participants' performance with a detailed rubric. All evaluators were trained by the researchers on the rubric before assessing participants. The rubric was handed from one evaluator to the next as the participant transitioned from station to station. Each participant had 7 minutes at each station before moving to the next station.

At stations 1 and 2 participants encountered SPs. The SPs were coached by 2 of the researchers with experience in training SPs. The SPs were used to promote consistency and accuracy during the simulation.^27,38 The SPs portrayed a college athlete who visited the athletic training clinic to obtain a cardiovascular assessment as part of the preparticipation physical examination. The evaluators for station 1 and 2 were ATs.

At station 3 participants encountered Harvey, the cardiopulmonary patient simulator, and the evaluator, a physical therapist. Participants were asked to place the stethoscope on the 4 heart valves and show the evaluator how they would perform auscultations of the heart.

At station 4 participants were asked by the evaluator, a physical therapist, to listen and identify the following heart sounds: (1) S3, ventricular gallop; (2) murmur; (3) S4, atrial gallop; and (4) normal S1 and S2. The audio CD by Salvator Mangione³⁹ was used to play specific heart sounds. The reason the audio CD was used rather than Harvey, the cardiopulmonary patient simulator, was because of the time constraint to complete the pretest. Furthermore, the simulation laboratory was able to provide only one cardiopulmonary patient simulator at the time of the CPE course.

Immediately after completion of the OSCE, the participant was sent downstairs to the computer laboratory to complete the 31-item multiple-choice knowledge examination. Support personnel aided in showing the participants their way while ensuring no information was shared among participants about the experience in the OSCE.

Multiple-Choice Examination

The objective of the 31-item multiple-choice examination was to assess the participants' knowledge on cardiovascular physiology and cardiovascular evaluation preintervention and postintervention. The examination was scored dichotomously (0 = incorrect, 1 = correct) and based on the 7 learning objectives provided in Table 2. The maximum possible score was 31. A higher score on the examination indicated greater knowledge. The examination was administered on the computer using the computer survey program Qualtrics (Provo, UT). The examination was written by a physical therapist who has been teaching cardiopulmonary physical therapy since 2002. A volunteer at the CEU event supervised the computer laboratory during the pretest and posttest to ensure participants completed the examination without browsing the internet for relevant information.

Intervention

Learning Session

After the pretesting concluded, all participants attended a 3-hour interactive instructor-led learning session designed by the aforementioned physical therapist with extensive experience in cardiopulmonary physical therapy. There was a 22:2 ratio of participants to instructors. The instructors were 1 physical therapist and 1 doctor of physical therapy student. The physical therapist took the lead during the lecture and the student instructor helped facilitate the interactive portions. The 7 learning objectives (Table 2) were addressed throughout the didactic lecture. The lectures were broken into 4 short (10–20 minutes) PowerPoint presentations (Microsoft, Redmond, WA). Between presentations, brief (10–20 minutes) laboratory practice sessions were incorporated to engage the participants in active learning experiences. Although the didactic portion covered all 7 learning objectives, the laboratory sessions were purposefully designed to follow the layout of the 4-station OSCE (Figure 2). We wanted the participants to practice the skills in the correct sequence to facilitate learning.

One-Hour Simulation Type (High- and Low-Fidelity) Interventions

After the learning session, all participants were informed of which simulation group (high- or low-fidelity) they were randomly assigned into. This study did not have a control group because research has identified simulation as a beneficial instructional strategy; research comparing simulation-based instructional strategies is recommended.⁴⁰

The high-fidelity group (12 participants) attended a 1-hour, instructor-led training session using Harvey, the cardiopulmonary patient simulator. The same physical therapist who wrote the knowledge examination and presented the learning session led the high-fidelity simulation group. The physical therapist ensured participant adherence and was available to answer any questions or help refine the skills learned in the previous learning session, including heart sounds. During the hour, participants were required to practice on the simulator and each other. Because the ratio of participants to simulator was 12:1, the simulator was used primarily for stethoscope placement and listening to the abnormal heart sounds. Everyone took turns listening and practicing on Harvey. To maximize practice, the history taking and the physical examination components were practiced on other participants. The instructor did not continue teaching but answered any questions and facilitated learning in cases when the group or individuals had questions.

The low-fidelity group (10 participants) was asked to complete a 1-hour self-directed computer screen–based training in a computer laboratory. A list of self-guided heart-sound Web sites was provided to each participant and it was requested that participants navigate through the provided resources at their own pace with headphones in their ears (Table 3). The low-fidelity group was supervised by one of the researchers to ensure participant adherence and to encourage the participants to keep listening to the sounds and reviewing the information on the Web sites. The Web sites focused primarily on reviewing the relevant history questions, physical examination findings, stethoscope location, and heart sounds. Four of the heart sounds were selected to be identified during the pretest and posttest. The posttest procedures were the same as the pretest procedures.

Table 3 A List of Self-Guided Heart-Sound Web Sites

View Fullscreen

RESULTS

A mixed-methods analysis of variance was performed to test for differences within and between the high-fidelity and low-fidelity groups. All descriptive statistics for the 3 outcome measures are located in Table 4.

Table 4 Analysis of Variance Summary Table for Knowledge, Clinical Skill and History Taking, and Heart-Sound Identification

View Fullscreen

Knowledge Outcome

Participants in the low-fidelity group scored 40.9% (12.7 of 31) at pretest and 52.9% (16.4 of 31) at posttest. Those in the high-fidelity group scored 47.1% (14.6 of 31) at pretest and 61.0% (18.9 of 31) at posttest (Table 5). Table 4 displays the within-participants contrasts of knowledge and the between-participants effects of knowledge and fidelity type. The within-participants analysis indicates that the main effect of time (pretest to posttest) improved significantly (P < .001). All participants improved their knowledge scores significantly. The interaction between the knowledge gained and fidelity type was not significant (P = .704). The between-participants analysis also indicates that there was no significant difference (P = .258) between the amounts of knowledge gained by the high- and low-fidelity groups.

Table 5 Descriptive Statistics for Knowledge, Clinical Skill and History Taking, and Heart-Sound Identification

View Fullscreen

DISCUSSION

This study set out to identify whether high- or low-fidelity simulation is more effective for reinforcing the knowledge and skills associated with a comprehensive cardiovascular assessment as part of a CPE course for ATs. According to our findings, which align with nursing,^18,41,42 medical,^20,43,44 and athletic training professional education,⁴⁵ high- and low-fidelity simulation are both effective in teaching that skill. The low pretest scores and significant learning gains of each participant make this a skill worth reinforcing to ATs as CPE. Because endorsed documents from the NATA, such as the building blocks and position statements, set the standard of care, CPE should exist to provide clinicians the opportunity to practice and review knowledge and skills specific to the evidence presented in these documents. The techniques that can save a patient's life should be routinely practiced. The relationship between perceived and actual knowledge of ATs performing emergency management skills has been found to be poor.^46,47 Although this study did not set out to investigate perceived versus actual knowledge, we observed that the participants' actual knowledge and ability to perform a comprehensive cardiovascular assessment were poor.

All participants' average pretest scores were low. Specifically, the diagnostic accuracy for detecting cardiovascular abnormalities, as measured by identifying normal versus abnormal heart sounds, was at 24.9% before the intervention. This finding is comparable to findings for other medical professionals, except cardiologists, who have been identified to have a diagnostic accuracy that ranges from 20% to 40%.^48–51 All around this skill is worth reteaching and reinforcing, as sudden cardiac death is still occurring in the patient population ATs work with. To narrow this gap, CPE with simulation is an effective method to implement.

There are a number of reasons that could explain the low pretest scores. According to Neil et al,⁴⁷ the knowledge gap, as identified in our study by the pretest and posttest scores, could be attributed to participants' lacking motivation to learn, neglecting the evidence-based published literature (thereby not knowing what is the most current evidence), or never initially learning the skill effectively within their professional education. Whatever the reason, the pretest helped identify a gap in their knowledge and skill. It helped the participants to recognize the need for CPE in this area. According to adult learning theory, recognizing a deficit before a learning experience increases effort and a greater purpose in participating in a CPE course.⁵² Not recognizing such a gap before selecting CPE can prevent clinicians from seeking the appropriate CPE and can put patients at risk when clinicians are called upon to perform the skill.⁴⁶ A pretest performance assessment is more useful than a self-assessment⁵³ in uncovering the participant's knowledge and skill deficit, making it clear what the clinician needs to practice.⁵²

Certainly, it is unrealistic to add a pretest before every CPE course. However, adding a pretest is recommended to identify the deficits and create learning environments that help clinicians apply and refine deficit and acquired knowledge and skills,⁵⁴ especially those that are either new or infrequently performed in clinical practice.^55,56 Our study can be used as a reference when implementing CPE with simulation and possibly a pretest. The high-fidelity simulation can be used as preseason CPE training. Low-fidelity simulation can be implemented throughout the year to maintain competence and periodically refresh the knowledge and skill required to be ready for any situation.

Although learning gains were significant for both types of simulation, the high-fidelity group had significantly higher end scores on the OSCE because of the active nature of the learning experience. This is consistent with existing literature.^40,57 Specifically, Butter et al¹² found that third-year medical students who engaged in a curriculum that combined simulation with opportunities to actively practice and thereby manipulate the simulated environment improved their heart-sound identification scores when performing a cardiac assessment. In our study, the high-fidelity group attained more skill because of the hands-on, active, and interactive nature of the 1-hour intervention, which positively influenced the acquisition of knowledge and skill.^55,56,58 This CPE course required ATs to think, act, and perform skills they are expected to execute in clinical practice. Each participant took the information and immediately had the opportunity to apply and refine it versus simply noting the presented information,⁵⁴ as commonly occurs with a lecture-based CPE course.

A concern with high-fidelity simulators is their associated cost; their prices range from moderate to high, primarily because of their sophisticated computerized ability.⁵⁹ Not only are the mannequins expensive, but there are also costs associated with maintaining and operating the technology. In addition to the monetary costs, the facilitator of the CPE course must invest time in learning about the effective use of simulation to complement the learning environment. The benefits may outweigh the costs when considering the positive outcomes a simulator can have specific to patient safety,^25,60 preparation of health care professionals,^18–20,44 and maintenance of knowledge and clinical skills.^61–63 Alternatively, as is well documented within the literature of athletic training, an effective simulation can use SPs.^{27,38–64,65}

The decision to use low- or high-fidelity simulation, both of which lead to significant learning gains, can be made based on available resources. We found that both fidelity types were effective. Our findings suggest that there are some topics, including the knowledge and skills specific to a comprehensive cardiovascular assessment, where low-fidelity simulators can be used to reinforce a skill to ensure the maintenance of that skill. Our findings were consistent with those of Bonnetain et al,¹⁹ who compared 2 groups of second-year medical students and their ability to transfer learning to a high-fidelity patient simulator. In that study,¹⁹ the control group practiced cardiac arrest procedures in a traditional laboratory environment and the experimental group trained on a computer screen–based multimedia simulator. The results indicated that the experimental group transferred more learning to the high-fidelity simulator when compared with the control group who practiced and trained in traditional education settings.

Low-fidelity simulation is better to implement than nothing. As our study used a computer screen–based simulator, there may be value in adding low-fidelity simulation to online CPE courses for ATs. Online CPE courses are a preference of ATs who have a busy schedule,⁶⁶ and this study identified that the knowledge and skill refined during a 1-hour low-fidelity simulation transferred to the OSCE, which included high-fidelity simulation. This can also be implemented as a refresher for ATs during the season after they initially learn the skill during preseason training. When and if possible, we recommend the inclusion of low- and/or high-fidelity simulation when developing CPE, and especially when teaching the clinical knowledge and skills of a comprehensive cardiovascular assessment. The main premise of CPE is to ensure the maintenance of knowledge and skills. This may be achieved with CPE that actively engages the participant in learning opportunities.

Developing this CPE course with simulation was an extensive process from a design perspective. Gathering the resources and developing a CPE course for already-certified ATs from diverse backgrounds was different from preparing a lecture to present at a CPE course. No research in health care education exists that discusses developing a CPE course that incorporates simulation. Resources are available specific to implementing simulation in professional education^45,67,68 as well as in postprofessional education.⁶⁹

LIMITATIONS AND FUTURE RESEARCH

This study was not without limitations. One limitation was the sample size; it was relatively small and does not adequately represent all ATs, which limits the generalizability of the results. The nonvalidated assessments are another limitation. Although the instruments were created by a panel of experts and reflected the content of the educational presentation and NATA Building Blocks for Cardiac Assessment, further validation would have strengthened the instrument. Another limitation was the instructor was available only to the participants who were assigned to the 1-hour, hands-on, high-fidelity group, leaving the low-fidelity group unable to ask content-specific questions to clarify their own understanding. The low pretest scores may be considered another limitation, although we find that low pretest scores are somewhat expected in educational research. The learning gains may not be interpreted as meaningful because the pretest scores illustrate that the participants did not know about or how to perform the skill being evaluated. This specifically holds true for the cardiac auscultation and heart-sound portion of this study. The other material, history taking, signs and symptoms of Marfan syndrome, and the cardiac cycle, is required to be instructed in accredited programs. Another limitation was the OSCE: the participants' true clinical performance may have been altered by the context of the OSCE, which may not have adequately mimicked a real-life scenario.⁷⁰

Future research should use larger samples that are more representative of all ATs. We should investigate the effectiveness of low-fidelity simulation and the extent to which the acquired knowledge and skill transfers to a real patient. We found that low-fidelity simulation was an effective method in teaching cardiovascular assessment, but future research should determine to what extent this knowledge and skill transfers to the effectiveness of the AT's clinical practice. This study did not investigate the retention of knowledge or skill; therefore, future research should investigate if retention is attained through a CPE course that uses simulation. Lastly, future research should compare the actual knowledge and skill, as done in this study, to perceived competency of performing the skill.⁴⁷

CONCLUSIONS

Maintenance of competence in clinical knowledge and skills is the responsibly of the AT. Our findings expand the literature and suggest that simulation-based interventions as part of a CPE course successfully led to the acquisition of clinical knowledge and skill. High- and low-fidelity simulation can be used with ATs, as part of CPE, to improve knowledge and skills specific to a comprehensive cardiovascular assessment. The low pretest scores and significant learning gains make this a skill worth routinely refreshing and teaching. Adding a pretest can be useful to identify knowledge and skill deficits before participating in CPE. Including simulation and time to actively practice and refine the content will narrow the knowledge and skill gap and develop better-prepared ATs.