Athletic Training Students Demonstrate Airway Management Skill Decay, but Retain Knowledge over 6 Months

Participants

Twenty-five athletic training students (8 male, 17 female; age = 21.12 ± 1.42 years) enrolled in the professional phase of undergraduate athletic training programs during the 2013–2014 academic year participated in the study. Participants were recruited from 2 separate undergraduate programs, from 2 separate states in the same National Athletic Trainers' Association district, and represented all levels of the athletic training program (10 sophomores, 6 juniors, 9 seniors). All participants were required to have taken an emergency responder or equivalent course wherein AM knowledge and skills were formally taught and evaluated by a qualified instructor. Demographic data are provided in Table 1.

Table 1.  Participant Demographics

View Fullscreen

Procedures

The institutional review boards at both sponsoring institutions approved this study. All participants engaged in baseline assessment procedures, including cognitive evaluation in the form of a knowledge assessment and clinical skills evaluation. In the week afterward, all participants attended a review session, which included an overview of the knowledge and concepts associated with AM, as well as an overview and rehearsal of the clinical skills. During the week after the review session, participants were assessed once again on the knowledge and skills associated with the use of airway adjunct devices (ie, OPA and NPA).

After the postreview session assessment (T1), participants were randomly assigned to either a control (n = 13) or experimental group (n = 12). To identify a time frame for when knowledge or skill decay may occur related to the use of both the OPA and the NPA, the experimental group was reevaluated after the review session at the following intervals: 1 month (T2), 3 months (T3), and 6 months (T4). The control group was simply reevaluated at the 6-month (T4) mark.

Assessment Procedures

Participants underwent both knowledge and skill assessment procedures at various intervals throughout the study. A baseline assessment was conducted in the week before the review session, and included a knowledge assessment in the form of a 12-item multiple-choice examination including 5 distractors, as well as a clinical skills assessment in the form of a practical examination. The purpose of the written examination was to assess cognitive knowledge related to the use of airway adjunct devices, including OPA and NPA, in an emergency situation. Participants' clinical skills were assessed by taking part in a practical examination related to the use of airway adjunct devices, which included patient and device preparation, proper sizing, and proper technique used to insert the device. Insertions of OPA and NPA devices were assessed separately. The clinical skills were performed on an airway-training manikin. The practical examination was scored using a skills checklist comprising 9 yes/no items for each skill (OPA insertion and NPA insertion). All items on the skills checklist were weighted equally, and scores included the percentage of correct responses. Both the written and practical examinations were developed from a recent article²⁰ related to the use of airway adjunct devices. Both assessment instruments were sent to 3 individuals with expertise related to emergency management and pedagogy to determine content validity.

After participants attended the review session, they were again assessed (T1) on the knowledge and skills associated with the use of airway adjunct devices, using the same assessment procedures. The purpose of this assessment was to compare results with the baseline assessments to ensure that the review session had the intended purpose of increasing knowledge and skill.

After the postreview session assessment (T1) procedures, participants were randomly assigned to either a control (n = 13) or an experimental group (n = 12). The experimental group repeated the assessment procedures (written and practical examinations) previously described at the following intervals after attending the review session: 1 month (T2), 3 months (T3), and 6 months (T4). The purpose of the frequency of this assessment was to establish a timeline for knowledge and skill decay (if one exists). The control group was only reassessed 6 months (T4) after attending the review session. The postreview session assessment (T1) was used to track knowledge and skill decay for both groups at each assessment interval. The timelines used in this study were consistent with those of previous studies evaluating knowledge and skill decay in the health care profession.^8,9,11–14 Figure 1 highlights the study methods and timeline. The same instruments were used to evaluate knowledge and skills in all assessment procedures throughout the study. The items on the multiple-choice written examination were randomized with each administration to avoid memorization. The 2 skills (OPA and NPA) that were assessed on the practical examination were also randomized with every assessment. Furthermore, the order in which each task (written or practical examination) was conducted was randomized with every assessment.

View Figure

• Download as Powerpoint
• Download Figure

Review Session

Once baseline assessment procedures were completed, participants were required to attend a review session, which was led by the investigators at their respective institutions. The review session included the use of a standardized lecture (PowerPoint presentation with recorded voice-over) that included the cognitive knowledge associated with the use of airway adjunct devices. A standardized video was shown that demonstrated the proper clinical skills associated with inserting an OPA and NPA. The same lecture and video demonstration was used at each research site to ensure that identical information was being conveyed to the participants. Participants were given a hard copy of the PowerPoint lecture materials, and were given a clinical skills review sheet that outlined the skills associated with inserting an OPA and NPA as defined by Berry and Seitz.²⁰ After participants viewed the lecture and both videos, they practiced and observed the clinical skills associated with inserting airway adjunct devices on an airway trainer manikin. While the participants practiced their skills, the investigator provided corrective feedback and assistance as necessary. Participants practiced the skills until they felt confident that they were proficient, based on investigator feedback, in the knowledge and skills associated with airway adjunct devices.

The review sessions were conducted in an athletic training classroom/laboratory setting. Participants were requested to refrain from studying the material or practicing their skills outside of the review session. The participants were reminded about these expectations on a regular basis.

Data Analysis

Descriptive statistics were used to calculate demographic data, AM knowledge, and clinical skills scores. Repeated-measures analysis of variance with between-participants (group) and within-participants (time) effects assessed pre-post changes in knowledge/clinical skills scores. Data from the combined groups was analyzed at baseline, T1, and T4 to identify overall changes in knowledge and skills over time. The Cronbach α determined internal consistency for the knowledge assessment. IBM SPSS Statistics for Windows (version 21; IBM Corp, Armonk, NY) was used to analyze the data. The α level was set a priori at P < .05.

Knowledge

The knowledge assessment demonstrated internal consistency with an α coefficient of .35 (minimal) at baseline and .526 (minimal) at 6 months (T4). Analysis revealed no differences in AM knowledge between the experimental and the control groups (F_2,46 = 0.37, P = .70) at any of the assessment intervals (Figure 2). The experimental group demonstrated a significant difference in knowledge scores over time (F_4,44 = 9.01, P < .001) where baseline knowledge scores (69.44 ± 12.97) were significantly lower than T1 (84.03 ± 10.33), T2 (80.56 ± 9.62), T3 (82.64 ± 9.03), and T4 (82.64 ± 7.50) knowledge scores (Table 2). There were no differences in posttest (T1) knowledge and all other assessment time frames. A significant difference (F_2,46 = 28.44, P < .001) was seen in the percentage of correct responses between the baseline and posttest for the combined groups, where baseline knowledge scores (65.33 ± 15.72) were significantly lower than T1 (82.00 ± 12.88) and T4 (79.33 ± 13.63) knowledge scores (Table 2). Although there was a trend toward knowledge decay at T4 for the combined groups, the mean differences were not significant between T1 and T4 knowledge scores (Figure 3).

View Figure

• Download as Powerpoint
• Download Figure

Table 2.  Knowledge and Clinical Skill Assessment Descriptive Data^a

View Fullscreen

View Figure

• Download as Powerpoint
• Download Figure

Clinical Skills

Oropharyngeal Airway

The OPA clinical skills assessment demonstrated internal consistency with an α coefficient of .61 (moderate) at baseline. There were no differences in OPA skills between the experimental and the control groups (F_1.51,34.61 = 0.40, P = .61) at any of the assessment intervals (Figure 4). The experimental group showed a significant difference in OPA skills over time (F_2.49,27.43 = 24.85, P < .001) where baseline skill scores (59.26 ± 14.47) were significantly lower than T1 (92.59 ± 10.94), T2 (89.81 ± 12.04), T3 (86.11 ± 12.65), and T4 (82.41 ± 12.94) skills scores (Table 2). More importantly, there was a difference in OPA skills noted between T1 (92.59 ± 10.94) and T4 (82.41 ± 12.94), indicating a decay in that skill after 6 months, but no differences in OPA skills between T1 and T2 or T1 and T3. A significant main effect for time on OPA clinical skills for the combined groups (F_1.50,34.60 = 65.02, P < .001) found baseline skill scores (56.00 ± 17.41) were significantly lower than T1 (88.88 ± 12.83) and T4 (81.33 ± 12.31) scores and T1 scores were significantly different from T4 scores (Figure 3).

View Figure

• Download as Powerpoint
• Download Figure

Nasopharyngeal Airway

The NPA clinical skills assessment demonstrated internal consistency with an α coefficient of .66 (moderate) at baseline. Analysis revealed no differences in NPA skills between the groups (F_1.62,37.32 = 0.630, P = .51) at any assessment interval (Figure 5). There was a significant difference in NPA skills scores (F_2.54,27.94 = 34.08, P < .001) in the experimental group over time. Specifically, baseline skill scores (44.44 ± 22.22) were significantly different from T1 (91.67 ± 8.38), T2 (88.89 ± 13.40), T3 (79.63 ± 20.01), and T4 (87.04 ± 10.42) skill scores (Table 2). Unlike OPA clinical skills, there was no significant difference noted between T1 and any other assessment time frame, indicating that NPA skill was retained in the experimental group. Scores for the experimental group demonstrated declines from T1 to T2 and from T2 to T3, but unexpectedly improved at T4. A significant main effect for time on NPA clinical skills for the combined groups (F_1.62,37.31 = 65.02, P < .05) indicated that baseline skill scores (44.00 ± 20.91) were significantly different from T1 (89.33 ± 9.34) and T4 (82.67 ± 14.01) scores, but that scores at T1 were also significantly different from those at T4 (Figure 3).

View Figure

• Download as Powerpoint
• Download Figure

Knowledge Retention

Our results indicate that AM knowledge is retained over a 6-month period of time (Figure 3). All participants in this study improved in their knowledge from baseline assessment to posttest, and cognitive knowledge was retained in both groups over a 6-month period of time. Both groups demonstrated subpar scores on the baseline assessment, which suggests a need to review the content for all students on a regular basis. It is important to note, however, that there was a trend toward knowledge decay in both groups at 6 months, but this decline in knowledge did not represent a large drop in scores and was not statistically significant. In this study, despite having repeated attempts, the experimental group did not perform better on the knowledge assessment at the 6-month interval.

Research in other health care professions is limited and results are mixed. One study¹⁶ in practicing physicians and nurses found that after 6 months of cardiopulmonary resuscitation (CPR) training, physician knowledge deteriorated to a pretraining level, whereas nurses maintained their knowledge over 12 months. Madden¹⁷ found that 72% of nursing students achieved a pass standard immediately after CPR, but only 44% achieved a pass standard 10 weeks after training. Likewise, 3 months after a trauma course, nurses failed to retain cognitive knowledge gained in the course.¹⁸

To the contrary, Smith et al⁸ found that nurses retained knowledge in both basic life support (BLS) and advanced cardiac life support (ACLS), which included AM. Notably, the nurses who indicated increased confidence performed better on their cognitive knowledge and their ability to perform BLS or ACLS.⁸ In addition, emergency medical technicians demonstrated knowledge retention in defibrillator pad choices and placement on children after 6 months of training.¹⁹ Clearly, the evaluation of knowledge retention associated with emergency medical procedures is conflicted. Even though the present study did not yield significant knowledge decay over 6 months, the trend toward decay is important, particularly given the fact that participants were formally instructed in the content on 2 separate occasions (emergency responder course as a required course within the respective athletic training programs and the review session as a component of this study). Furthermore, the fact that the experimental group had repeated exposure to the knowledge assessment and still trended toward decay is consistent with research by Su et al,¹⁵ who found that paramedics, despite improved knowledge scores after a pediatric resuscitation course, had significant cognitive knowledge decay at 12 months even though 75% of study participants were exposed to an educational intervention at 6 months.

Skill Retention

The clinical skills associated with our participants' OPA and NPA skills decayed in the combined groups over a 6-month period of time (Figure 3). The purpose of having an experimental group, who underwent assessment at more frequent intervals, was to identify a timeline for skill decay, should the presence of decay exist. We found that OPA or NPA skills did not decay more rapidly than at a 6-month interval. The OPA skills did steadily decline at each assessment interval (1 month, 3 months, 6 months), but the differences were not significant until the 6-month interval. The NPA skills of the experimental group, on the other hand, also trended toward skill decay, with a sharp decline most notable at the 3-month assessment interval, followed by an unexpected increase in score at the 6-month assessment interval. In contrast to the OPA skill, these results were not statistically significant at the 3-month or the 6-month interval. However, when the groups were combined, it was found that both OPA and NPA skills demonstrated significant skill decay at 6 months. Despite having multiple attempts over the 6-month test period, the experimental group did not perform OPA or NPA skills better than the control group. Baseline assessments revealed that both groups performed poorly on OPA and NPA skills, which indicates the need for students to rehearse these skills on a regular basis.

As with any psychomotor skill, the lack of adequate and timely practice of the skill leads to decay over time.^7–14 The present study echoes findings in other health care professionals as it relates to skill decay of acute care skills. In nurses, clinical skills associated with BLS skills dropped from a 100% pass rate to a 66.7% pass rate just 3 months after the initial training.⁸ Furthermore, this same study found that after 9 months after initial training, the pass rate declined from 92.9% to 35.7%.⁸

Likewise, research examining CPR skills of first responders¹⁰ and endotracheal intubation skills in paramedics⁹ shows patterns of skill decay as early as 30 days after initial training. Anderson et al¹⁰ found that CPR skill retention of first aid attendants within the workplace began deteriorating less than 30 days after the training. Kovacs and colleagues¹¹ found that AM skills of health science students declined after the initial teaching of the skill, decreasing by 25% over a 10-month period, with the sharpest decrease occurring in the first 16 weeks. In inexperienced paramedics, Ruetzler et al¹² found that endotracheal intubation skills declined from a 78% success rate to a 58% success rate after 3 months. After initial training, 100% of first responders demonstrated competence in the application of extrication cervical collars; however, only 61% maintained this competence after 12 months.¹³ Additionally, McKenna and Glendon²¹ showed a 50% decline in CPR skills over a 2-month period.

The present study reinforces previous findings in other health care professionals that skill decay occurs in the athletic training acute care competencies. The deterioration in OPA and NPA clinical skills should not be equated with a total loss of skills. Mean scores averaged over 80% for the combined groups at the 6-month assessment interval. Therefore, even though the skills decayed over a period of time, some components of the skills were maintained.

Studies that involved the deliberate practice of skills on a regular basis showed less decay of skills.^9,11,14 Hein and colleagues²² engaged paramedic students in a simulation experience at 6 months after initial instruction on airway insertion skill and found that participants retained skill better than those who did not participate in the simulation. Kovacs et al¹¹ found that independent skill practice plus instructor feedback at regular intervals was effective in maintaining clinical skills associated with AM. Advanced cardiac life support skills were retained over 14 months by medical residents who engaged in deliberate practice and small group simulation-based educational interventions.¹⁴ Broomfield²³ showed that a refresher CPR course showed initial improvement in CPR skill, but then 10 weeks later those skills again deteriorated.

Athletic training educators may consider the value of regular educational interventions to combat the occurrence of skill decay in acute care skills. Many programs engage in annual in-service programming before the start of the academic year. Acute care skills could be incorporated into this annual in-service program, with a skills refresher 6 months later. Certainly, supervised laboratory activities addressing the acute care skills that are required in regular intervals (eg, every 6 months) are another way to combat the effects of skill decay. Some programs incorporate simulations into their curriculum, and simulations that involve the acute care skills could be another mechanism to incorporate rehearsal of these skills. Lastly, clinical integration proficiencies or activities that would require the practicing of acute care skills with preceptors in the clinical education setting, which would benefit both the preceptor and the student, are another mechanism to ensure these skills are practiced regularly to combat the effects of skill decay.