Retained Knowledge and Use of Evidence-Based Practice Concepts

KEY POINTS

• Athletic trainers retained foundational knowledge of evidence-based practice concepts but did not implement this knowledge into clinical practice.

• Foundational knowledge in evidence-based practice concepts has not improved over the past 5 years.

• Athletic trainers should evaluate continuing education opportunities and enroll in evidence-based practice programming that best targets their personal knowledge and practice needs.

INTRODUCTION

Due to the growing emphasis on improving the quality of health care delivery in the United States, evidence-based practice (EBP) has been highlighted as an inherent component of nearly all aspects of health care education and practice.¹ In accordance with this emphasis, athletic training has included EBP concepts both in professional education programs² and as requirements for continuing professional education (CPE).³ Research has identified that athletic trainers (ATs) value EBP, yet do not possess the knowledge to implement associated concepts into clinical practice.⁴ Currently, the Board of Certification (BOC) requires all ATs to obtain 10 continuing education units (CEUs) in the EBP category every 2 years.³ The associated EBP classifications include Foundations of EBP and Clinical EBP programming. Foundations programs emphasize location, evaluation, and application of evidence, whereas Clinical programs promote presentation of clinically relevant topics formatted around a clinical question.⁵

While the BOC has implemented these requirements, and ATs are enrolling in such programs, little is known regarding the outcomes of CPE programming in these formats. Also, there is a dearth of information regarding how ATs access EBP resources. The purpose of this project is to present follow-up findings regarding knowledge, confidence in knowledge, and use of EBP concepts 1 year after attendance at a 5-hour Foundations of EBP workshop. Additionally, perceived barriers and envisioned use of EBP concepts will be identified.

METHODS

Participants

We invited a convenience sample of ATs who attended a BOC-approved EBP Foundations category workshop to participate in this study at 3 time points (preworkshop, immediately postworkshop, and 12 months postworkshop). The workshop was held at 2 Division I universities in Districts 2 and 3 of the National Athletic Trainers' Association (NATA) and was attended by 149 ATs; all attendees were invited to participate at each time point. Of the 149 attendees, 123 ATs completed both pre- and immediately post-EBCKAU surveys. Consenting participants were excluded from analysis if they did not fully complete the survey at any of the 3 time points. Of the 123 initial matched participants, 27 ATs continued participation to the 12-month time point, for a 22% response rate. Figure 1 portrays the study design and attrition throughout data collection while Table 1 portrays demographic information for all participants.

View Figure

• Download as Powerpoint
• Download Figure

Table 1.  Participant Demographics

View Fullscreen

RESULTS

Perceived Knowledge and Confidence in Knowledge

A repeated measures analysis of variance with a Huynh-Feldt correction determined that mean knowledge scores differed significantly between time points (F_2.0,52.0 = 18.91, P < .001). Post hoc tests using the Bonferroni correction revealed a significant increase in perceived knowledge from pre- to immediately postworkshop as well as pre- to 12 months postworkshop. Table 2 provides the mean and SD for knowledge scores at all time points.

Table 2.  Repeated Measures Analysis of Variance Results for Knowledge Scores

View Fullscreen

The MDC value for the EBCKAU was identified as 1.65 or the equivalent of 15% of the 11-question survey. In this sample, preworkshop participants had a mean knowledge score of 55%, correlating to low initial knowledge of EBP concepts; immediately postworkshop this value increased to 74%, which exceeds the MDC value, thus indicating significant knowledge gains. At 12 months postworkshop, participants had a mean knowledge score of 66%. While this value represents a decrease in total score from immediately posttest, it is not significant (P = .074), nor does it exceed the MDC value, thus demonstrating no clinically relevant difference in knowledge from immediately postworkshop to 12 months.

On the 4-point confidence scale, confidence in knowledge was significantly different over time (z = −4.55, P < .001). From preworkshop (median = 28) to immediately postworkshop (median = 37), an increase in confidence was seen, whereas a significant decrease in confidence in knowledge was seen from immediately postworkshop to 12 months postworkshop (median = 29). Table 3 provides an overview of all values related to the confidence-in-knowledge analysis. There was no correlation between postworkshop knowledge and confidence at any of the 3 time points, and no significant relationships were identified between years of BOC certification, years of clinical practice, or age and knowledge or confidence in knowledge. Additionally, no difference was seen in EBP use as participants showed no significant difference in the likelihood to use clinical question construction, compilation research, incorporation of PROMs, and subscription to literature services, before or after the workshop (Table 4). Participants identified specific resources they use 2 times per week or greater as the Internet, previous clinical experience, and peer-reviewed research as the most often utilized items (Figure 2). Figure 3 provides an overview of the ranking of resources used most often to identify evidence for use in patient care.

Table 3.  Wilcoxon Signed Ranks Results for Confidence in Knowledge

View Fullscreen

Table 4.  Participant Implementation of Evidence-Based Practice Concepts in Clinical Practice

View Fullscreen

View Figure

• Download as Powerpoint
• Download Figure

View Figure

• Download as Powerpoint
• Download Figure

Participants reported perceived barriers to use of EBP as time and resources. Time was recorded by many ATs (>50%) as a barrier to their use of EBP. Specific to resources, participants cited as barriers availability, cost and accessibility to some literature sources, and lack of awareness of how to access other resources. Envisioned use of EBP responses was classified into the themes of patient care and a mechanism to improve knowledge. Regarding patient care, ATs responded that they might use an EBP approach when adjusting rehabilitation plans, to supplement their decisions with their own clinical expertise, and when needing to evaluate a more effective treatment for a patient. In the area of using EBP as a mechanism to improve knowledge, participants referenced the BOC EBP category requirement as an area of use, as well as to have better knowledge of how to use/incorporate scholarly evidence. Figure 4 depicts the themes identified for use of EBP concepts.

View Figure

• Download as Powerpoint
• Download Figure

DISCUSSION

As the athletic training profession has seen an increase in the number of available CPE offerings that focus on EBP concepts over the past 7 years, knowledge has improved among those ATs who have enrolled in such courses.^7,10,11 In addition to the workshop presented here, specific CPE offerings include Web-based modules and other short courses. For example, the NATA Web-based learning modules for EBP became available to the membership in 2011 and include resources and instructions on multiple concepts pertinent to the EBP process (http://www.nata.org/career-education/education/ceu-info/ebp). Assessment of knowledge among ATs who completed these modules determined that this was an effective mechanism to enhance EBP knowledge, with most participants increasing their knowledge score by at least 6 points out of a possible 60-point maximum score.¹¹ Similarly, an evaluation of a 5-hour workshop related to EBP concepts found a small knowledge increase from 66.0% to 69.5%.¹⁰ While these findings are of note, both studies were conducted before the implementation of the BOC EBP category requirement for CEUs, and neither assessed knowledge retention over a longitudinal period of time.

As of July 2017, the BOC Credentialing Program has approved more than 1100 programs with EBP category designation (J. Roberts, e-mail communication, July 2017). Of these programs, only 123 have received the designation of Foundations programs, while more than 1000 have been designated as Clinical programs. As 1 full cycle of EBP reporting has passed, and a knowledge gap is still evident for Foundations of EBP content, more programs are needed to educate ATs on how to understand foundational EBP concepts in order to be better consumers of clinically based information.⁷ In addition to increasing these offerings, ATs are encouraged to self-assess their current knowledge levels of EBP and enroll in courses that will help to enhance their EBP foundational knowledge.

Within the current study, ATs who enrolled in the workshop increased their perceived knowledge in EBP concepts from pre- to postworkshop and demonstrated no significant change in that knowledge over the next year. A change in knowledge score of 1.65 (MDC) was needed to be confident that knowledge change had occurred. This value was exceeded from pre- to immediately postworkshop, indicating knowledge gains related to EBP. While there was a decrease from immediately after to 12 months after workshop, it did not exceed 1.65, and therefore does not demonstrate a significant loss of knowledge over this time frame. While the advancement of knowledge is important, the actual implementation of EBP concepts in clinical practice appears to be a larger problem. These findings, combined with the fact that ATs' confidence also saw an increase from pre- to postworkshop, with a subsequent drop in confidence over the next 12 months, provide evidence that ATs are not actively changing their behavior to utilize learned EBP concepts in their day-to-day practice. In general, it is not surprising that ATs are not implementing concepts that they are not confident in over time.

Specific to the lack of reported behavior change as a result of this workshop, the use of compilation research and implementation of PROMs, for example, were no more likely after the workshop than they were before. This lack of incorporation may have resulted from several factors. First, it is possible that the length and level of active immersion in workshop content may have limited the ability to incorporate EBP concepts in clinical practice. Stevenson et al¹² in 2004 suggested that behavior effect may be related to intervention intensity. Specifically, these authors theorized that programs more substantial and rigorous than a short-course workshop may have more consistent effects on behavior change. Rather than lecture-only programming, future CPE offerings may consider engaging participants in more application-based instructional techniques that encourage discussion and guided self-assessment for potential implementation of these concepts.

Next, the theory of planned behavior,^13,14 or the intention to actually perform behavior, may not have been evident in this sample, thus resulting in limited behavior change regarding EBP concepts. This theory is grounded in the intention to perform a behavior and involves 3 facets: (1) the attitude toward the behavior, (2) the level of importance of the behavior, and (3) the ability to control or demonstrate the behavior.¹⁵ For example, the reported lack of use of PROMs both before and after workshop attendance is similar to findings of Hankemeier, Popp, and Walker,¹⁶ who reported in 2017 that most ATs are unfamiliar with PROMs and are not using them in clinical practice. These authors postulated that their results were influenced by the theory of planned behavior in that educational strategies that influence knowledge do not necessarily ensure application to practice.¹⁶ With regard to the current study's educational strategy, which featured a stated workshop objective of instruction in PROM totaling approximately 60 minutes, ATs were no more likely to use PROMs after the workshop than they had been before the workshop. Educational strategies should continue to target increased knowledge of PROMs, but also transition to an emphasis on intent to implement these tools after training.

The behavior of incorporating PROMs does not seem to have changed over recent years. In 2014, researchers identified that only 26% of ATs utilize these instruments in patient care,¹⁷ which is nearly identical to the findings of this study of 22% preworkshop and 29% postworkshop. Researchers have also found that 71% of ATs' employers do not require the use of PROMs and further stated that they would be more likely to use them if their employer supported their use.¹⁶ Another study reported that 46% of ATs who did use PROMs did so because of employer mandates, while another 31% were encouraged to do so by their employers.¹⁸ As PROMs are a vital component of patient-centered care and EBP, these combined findings are of note, as many ATs are missing a vital piece of objective patient data.

RECOMMENDATIONS

The current approach of education followed by hopeful implementation does not appear to be successful. From an administrative standpoint, there needs to be an increase in support and resources to encourage the use of clinical outcomes and EBP. Consideration of a cultural shift to mandate and support use of these tools may be beneficial to accomplish behavior change. The Australian Physiotherapy Association, for example, mandated the use of valid outcome assessments as this was considered the standard of care.¹⁹ Through this mandate, the Association provided education, workshops, and tools to assist clinicians in meeting the new demands, which resulted in a greater than 35% increase in implementation of patient outcome measures. Through training, allocation of resources, and support, employers can increase the implementation of patient outcomes and EBP concepts.²⁰ Whereas the inclusion of required EBP continuing education by the BOC is a start, additional organizational emphasis mandating EBP implementation from the NATA may be a next viable step.

While implementation of EBP concepts is recommended, it can be difficult due to limited resources and established mechanisms to overcome barriers.²¹ Findings related to these aspects of EBP use among this population of ATs were similar to those of other studies.^4,7,22,23 Specifically, ATs continue to report the use of Internet sources, library databases, professional literature,²³ and their own clinical experience and that of others as resources utilized more than 2 times per week for patient care–related activities. Of note in this study is the fact that the Cochrane Collaboration, one of the most rigorously developed sources of evidence, was again not among the top reported resources utilized by participants. This lack of identification of the Cochrane Collaboration is similar to findings among ATs²³ and other health care professionals.²⁴ It should be mentioned that the Cochrane Collaboration (http://us.cochrane.org/) may be inaccessible to professionals outside medical and university settings, thus resulting in decreased use among ATs in comparison to other more mainstream resources. Considering barriers, participants in this study identified that knowledge of and access to resources remain as obstacles to the incorporation of EBP concepts. As these findings are identical to those in previous athletic training literature,^11,22,23,25 it remains necessary to provide ATs with access to quality evidence-based resources.⁴ Although Keeley et al²⁶ suggest that adequate resources do exist for ATs, there should be better education on how to access those resources. As we move toward decreasing the gap in EBP resource awareness and toward broader implementation of EBP, CPE offerings should continue to provide programming that identifies quality resources available and how to access such resources. ATs are encouraged to expand their use of available resources to establish a larger body of evidence and determine which facets best match their clinical practice.

As the current Athletic Training Educational Competencies require education in EBP and PROMs,² a knowledge gap exists between younger ATs who have received this training as part of professional education and more experienced ATs who have not. This study-associated workshop aimed to narrow the knowledge gap; however, this outcome was not fully realized. Considering that participants in this study did improve perceived knowledge of EBP concepts and indicated an intention to use PROMs immediately postworkshop but ultimately did not change their incorporation of foundational EBP concepts into clinical practice, we know that there is still much work to do. The initiatives set forth by the BOC, including the requirement of EBP-focused education credits and the pilot portfolio system,²⁷ help to target factors that facilitate the implementation of EBP. These factors include (1) identification of the value of EBP, (2) the attainment of EBP knowledge and skills, and (3) establishment of a culture to support EBP including access to evidence,^26,28–30 and may be valuable for consideration by ATs in future efforts to incorporate EBP concepts.

LIMITATIONS AND SUGGESTIONS FOR FURTHER RESEARCH

This study does have limitations that should be considered. The small sample size of ATs that continued at all time points of this study is of note. It is possible that the participants who elected to continue through the entire study had a more vested interest in the findings, and thus were more inclined to participate. While the sample size is small, this is one of the few published articles in athletic training that examine EBP-related knowledge and confidence in knowledge longitudinally. Another limitation is that we were unable to objectively measure direct implementation of workshop-related knowledge, and instead relied upon participant self-reported information. Future research should aim to focus on strategies to improve not only knowledge but also implementation of EBP concepts and ultimately determine if use of EBP has an effect on patient outcomes. Additionally, research may be conducted related to behavior change and the role of administration or employer in mandating the use of EBP concepts and/or PROM during patient care.

CONCLUSIONS

In this study, ATs appeared to increase EBP knowledge after a 5-hour workshop, with no significant change in that attained knowledge over the next 12-month period, while their confidence in this knowledge significantly decreased. Despite this increase in knowledge after the workshop, it appears these ATs were reluctant to change their approach to clinical practice. Given the overall emphasis on EBP in health care, the BOC requirements for EBP continuing education, and the still apparent knowledge gap regarding EBP concepts, ATs should perform self-assessment and consider enrolling in CPE courses that will target and improve their knowledge and confidence related to these topics. The reported perceptions of ATs regarding barriers to, resources for, and use of EBP can be reflected on for guidance in the creation of content for future CPE programming. Furthermore, consideration of an administrative- or profession-wide focus on the implementation of EBP to further improve ATs' patient care should occur. The implementation of EBP knowledge and skills will be paramount to furthering ATs' standing among other health care professionals.