Athletic Trainers' Current Knowledge and Envisioned Use of Foundational Evidence-Based Practice Concepts

Study Design

A mixed-methods, within-subjects design with preintervention and postintervention evaluation of ATs' EBP perceived knowledge via the Evidence-Based Concepts: Knowledge, Attitudes, and Use (EBCKAU) survey.¹¹ The study was approved by the university institutional review board of the primary researcher.

Participants

The EBCKAU was administered to a convenience sample of ATs who attended a 5-credit BOC-approved EBP CPE workshop entitled “Understanding Evidence-Based Practice and Patient-Reported Outcomes.” This 5-hour workshop, offered exclusively to ATs, was held during spring 2015 at 2 Division I universities in District 2 and 3 of the NATA and was attended by 149 ATs. Advertising of the workshop was sent to local medical organizations and athletic training professionals as well as alumni and affiliated AT preceptors of each university.

Of the 149 attendees, 123 ATs consisting of 63 men and 59 women (1 participant elected to not disclose sex) completed the EBCKAU at both preworkshop and postworkshop time points for an 83% participation rate. Consenting participants were excluded from analysis if they did not fully complete the survey on either the preintervention or postintervention administration. Table 1 portrays demographic information for all participants.

Table 1.  Participant Demographics

View Fullscreen

Instrumentation

The EBCKAU survey was administered online and has been previously deemed valid and reliable among a student population learning foundational concepts of EBP.¹¹ Because the participants in this study were considered students in an EBP workshop, the instrument was deemed appropriate for the population. Among this certified AT population, the knowledge section of the EBCKAU demonstrates a Kuder-Richardson (K20) value of 0.435. Ordinal values of the EBCKAU have an established Cronbach α of 0.70. Although the K20 value of 0.435 is not high, it does take into account the increased level of difficulty of EBP questions for novice foundational EBP course attendees, thus contributing to the consistency and reliability of the instrument.

Eleven multiple-choice questions were used to assess knowledge of the foundational steps of the EBP process and patient-reported outcome measures (PROMs). After each knowledge question, confidence in knowledge was rated on a Likert scale by asking participants to respond to the following statement: “I am ___ confident that I answered this question correctly,” with one of the following Likert scale responses: (1) not at all, (2) mildly, (3) moderately, and (4) extremely confident.

Participant perceptions of EBP were assessed via questions aimed toward describing ATs personal practice and resources for, barriers to, and envisioned use of EBP. The personal practice–oriented questions included ranking, checklist, and open-ended formats targeting how ATs access and use EBP resources. The ranking question asked participants to list and rank literature-search tools they use in order from most preferred (1) to least preferred (5). In addition, each participant was given a checklist and asked to identify which resources he or she uses more than 2 times per week when determining patient treatment plans; possibilities for this list included 13 options of intrinsic factors such as personal experience, extrinsic factors such as patient feedback, educational materials including previous course notes, published evidence, and colleague discussion. Open-ended questions were also included and permitted participants to describe their perceived barriers to the use of EBP concepts in clinical practice as well as ways in which they envision using EBP skills in future athletic training practice. Demographic questions aimed to describe the sample and determine representation of the population.

Intervention

This 5-hour BOC-approved Foundations of EBP educational program was designed by 3 athletic training educators considered to be experts in EBP. A primary aim for this workshop was to increase attendee competency in knowledge and application of EBP. Through the EBP techniques instructed in this workshop, participants learned how to establish clinically based questions specific to patient care via the patient, intervention, comparison, outcome (PICO) method; search the literature for information relating to the patient; critically appraise information; and apply information appropriately to clinical problems including selection and implementation of PROM. Specific examples and resources for clinical application, as well as a review of the importance of PROM athletic training clinical practice, were included. The structure of the course included lecture, guided practice, discussion, and group analysis.

Procedures

The EBCKAU survey was administered online (Survey Monkey, 2015 version; Survey Monkey Inc, San Mateo, CA) both before the associated workshop intervention and within 48 hours afterward. Delivery of the workshop was conducted by the primary research team with identical content at each workshop. Workshop location 1 (March 2015) featured 3 speakers considered experts in evidence-based practice, whereas workshop location 2 (May 2015) featured 1 of the original speakers presenting all information; instructional content remained consistent between locations and was presented in accordance with initial presentation of the workshop. Delivery of the workshop occurred over a consecutive 5-hour period at both locations.

Two weeks before the presentation of each workshop, the primary investigator (S.M.) sent an e-mail to registered attendees detailing the overview of the workshop, purpose of the study, invitation to participate in the research study, and link to the EBCKAU survey. Participants who had not completed the survey online before the workshop were given the opportunity to complete the survey on-site using either an online device or paper format. Immediately after the conclusion of the workshop, participants were e-mailed a link to complete the online postworkshop survey. Participants self-selected a unique identification number based on their initials and birthdate to track completion of the survey between preworkshop and postworkshop time points. The majority of participants completed the survey, via the online link, on-site before receiving their continuing education credits. The paper option was made available immediately postworkshop as well. Surveys completed in paper form (n = 5) were manually entered into the online platform by the research team. A follow-up e-mail was sent at the conclusion of the workshop requesting completion of the postworkshop survey from those who did not complete the survey on-site.

The survey included an introductory page with informed consent followed by the EBCKAU survey. Although all workshop attendees were required to complete the survey before and after to confirm attendance and serve as verification of completion for continuing education credit, the first question of the survey allowed the participants to opt into or out of the research study. Participants were given this opportunity on both preworkshop and postworkshop administrations of the survey.

Data Analysis

Data analysis was conducted at the conclusion of both workshops using SPSS Base for Windows, version 22.0 (IBM Corp, Armonk, NY). Descriptive statistics including means, standard deviations, and frequency values were analyzed and a Shapiro-Wilks test was conducted to support the assumption of normality. Knowledge scores were tabulated by giving a score of 1 to each correct answer and then summing all correct responses, for a maximum score of 11. After summation, a paired t test was conducted to determine differences in knowledge scores from preworkshop to postworkshop EBCKAU survey administrations.

Confidence in knowledge scores were attained by taking the score that correlated to the Likert response for each statement and then summing the 11 individual scores for an overall possible confidence score of 44. A higher score indicated higher confidence in knowledge. Wilcoxon matched-pairs signed ranks (T) were used to determine differences in confidence in knowledge. Pearson product moment correlations (r) were used to establish relationships between knowledge change scores and participant factors including prior EBP workshop experience and level of degree. Spearman rank correlations (ρ) assessed the relationship between postknowledge scores and post–confidence in knowledge scores. Statistical significance was set a priori at P = .05. Ranking and checklist questions were analyzed through frequency counts.

Due to the focused context of questions used in this survey (perceived barriers and envisioned use of EBP), open-ended responses were analyzed qualitatively through inductive content analysis to condense the raw data into brief summary format.^22–23 One researcher, with 7 years of qualitative research experience, performed the initial textual coding by assigning conceptual labels to all responses. Thematization then occurred as coded concepts were placed into common subthemes. As the data analysis process advanced through continued review of the data, subthemes were reorganized, as appropriate, to establish higher-order themes. Reevaluation and reorganization continued until all appropriate data had been thematically categorized into higher-order themes and subthemes.^22–25 After this thematic determination, the second researcher, who also has qualitative experience, served as the peer debriefer and conducted content analysis on the basis of the initial researcher's findings. Any discrepancies between researchers were discussed and resolved. This qualitative analysis process is consistent with other qualitative strategies.^26–28

Member checking and the previously described peer review process contributed to data triangulation, thus establishing trustworthiness of the data.^22–23,26 Member checking occurred after open-ended responses were retrieved and coded; the researchers shared the results with a convenience sample of consenting participants to member check themes and categories for accuracy and clarity. In addition, triangulation of sources was accomplished because the EBCKAU was administered as pretest and posttest, allowing for further corroboration of findings.^22,26

Perceived Knowledge and Confidence in Knowledge

Of the 149 attendees at the workshops, 123 completed the preworkshop and postworkshop EBCKAU surveys and consented to use of their responses, for an 83% response rate. For the knowledge scores, the Shapiro-Wilks test of normality indicated nonnormality (P = < .001); however, all other indices suggested that normality was a reasonable assumption including skewness (−0.143), kurtosis (0.526), histogram, and Q-Q plots. There was no significant difference in preworkshop or postworkshop knowledge scores between workshop locations (Table 2). Before workshop attendance, participants had a mean knowledge score of 58%, demonstrating low knowledge of EBP concepts; this value increased to 77% postworkshop, indicating moderate knowledge of EBP concepts. On average, knowledge-change scores were equivalent to an increase in total score by 2 points with a range of −3 to 7; 59% of participants increased their score by 2 points or more. Table 3 depicts preworkshop and postworkshop knowledge and confidence in knowledge results for all participants. Confidence scores improved from a 2.35 (mildly confident) to 3.38 (moderately confident) on the 4-point confidence scale. In addition, postworkshop knowledge was found to positively correlate to postworkshop confidence with a weak to moderate relationship (ρ = 0.408, P < .001). No significant relationship was identified between age, years of BOC certification, years of clinical practice, or hours of internet use and either knowledge or confidence in knowledge scores.

Table 2.  Knowledge Scores by Location

View Fullscreen

Table 3.  Evidence-Based Concepts: Knowledge, Attitudes, and Use (EBCKAU) Knowledge Scores

View Fullscreen

Resources for EBP

Both preworkshop and postworkshop, participants were provided a checklist of resources to identify which items they use more than twice per week when determining treatments for their patients. The resultant identified items were similar from preworkshop to postworkshop due to the short time that elapsed between assessments; therefore, only postworkshop findings are presented. After the workshop, the most often identified items were the internet, previous experience, coworker conversation, and discussion with other health care professionals. Figure 1 displays a frequency count of the resources identified by participants.

View Figure

• Download as Powerpoint
• Download Figure

At both survey-administration time points, participants were also asked to list and rank in order, from most used to least used, up to 5 databases/sources they use most often when conducting literature searches. Of the responses recorded by participants postworkshop, the most commonly reported databases/resources were Medline/PubMed, Google Scholar, and EbscoHost, collectively accounting for approximately 66% of all responses. Figure 2 shows the hierarchal rating of each commonly used database.

View Figure

• Download as Powerpoint
• Download Figure

Barriers to EBP

Participants were asked to identify their perceived barriers to the use of EBP in clinical practice via open-ended questions both preworkshop and postworkshop. All 123 matched presurvey and postsurvey participants provided these responses, and thematic trends were consistently identified at both time points in the areas of time, accessibility of evidence, knowledge related to EBP, applicability of evidence, and the culture of their practice environment. Figure 3 provides an overview of themes and supportive quotes for each theme.

View Figure

• Download as Powerpoint
• Download Figure

Envisioned Use of EBP

In an attempt to further define the value of the workshop, ATs were asked both preworkshop and postworkshop to identify how they envision using EBP concepts in future athletic training practice. Responses were distributed among 3 common themes before the workshop: patient care, knowledge of EBP, and teaching of EBP. After the workshop, participant responses demonstrated continued envisioned use in these 3 theme areas; however, an additional theme of envisioned incorporation of the EBP process within AT practice was identified. Within the theme of patient care, a new category was also delineated for incorporation of PROMs and justification of care. Figure 4 depicts the conceptual framework of themes and categories for both preworkshop and postworkshop responses.

View Figure

• Download as Powerpoint
• Download Figure

Patient Care

Within the patient care theme, 4 categories were established to further delineate AT responses related to envisioned use of EBP and included approach to patient care, to determine patient outcomes, and to assist in selection of interventions, as well as to assist in clinical decision-making. After the workshop, more specific responses related to the incorporation of patient-reported outcomes and justification of care were identified. Table 4 provides select supportive quotes for these categories from preworkshop and postworkshop responses.

Table 4.  Selection of Supporting Quotes for the Theme of Patient Care

View Fullscreen

Evidence-Based Practice Process

A novel theme was identified postworkshop as ATs provided envisioned-use responses that reflected specific workshop content related to the EBP process. This theme was further delineated into categories of the PICO process, access to resources, and an envisioned goal of incorporation of clinical expertise; each of these categories are considered to be steps of the EBP process. Figure 5 presents select participant quotes for this theme and associated categories.

View Figure

• Download as Powerpoint
• Download Figure

Knowledge and Confidence in Knowledge

Because the 2015 BOC reporting cycle designated the requirement of 10 EBP category CEUs per 2-year reporting period,⁶ programming is needed to provide varied and applicable BOC-approved topics to athletic training professionals. According to the BOC Credentialing Program coordinator, as of May 2016, there are approximately 725 unique programs approved in the EBP category. Of these approved programs, 99 are approved as Foundations programs and more than 580 are approved as Clinical programs. After implementation of the new BOC guidelines for approved providers in January 2016, the number of approved providers has decreased from approximately 1200 to 510. This decrease in providers has also led to an approximate 10% decrease in the number of Foundations-category programs available for CPE (e-mail communication, May 2016). Although more clinical than foundational programs have been approved, it is imperative that clinicians understand the foundational concepts and have a strong EBP knowledge base in order to properly understand and apply information obtained from clinical EBP programming. Because clinical EBP programming requires a focused clinical question, evidence of search strategy, and appraisal, ATs may develop a better understanding of, and appreciation for, EBP-approved clinical programming by having increased knowledge of foundational concepts. Because the first 2-year reporting period for this requirement closed in December 2015, a future need exists to establish more approved foundational EBP programs to allow ATs to gain more EBP-category CEUs without repeating courses.

Although athletic training knowledge is improving at a similar rate today to that of the last 5 years,^{10,12,13–16} a knowledge gap still exists and there is a need for foundational EBP CPE programs. Given that EBP became a required portion of the NATA Athletic Training Education Competencies⁴ in 2011, there is still a large portion of the practicing athletic training population that has not received formal education in foundational EBP content. Continuing professional education programs should continue to target this gap and offer programming that reviews foundational components of EBP that could translate to better understanding and appreciation of clinically based CPE programs.

Participants reported a mild level of confidence in EBP knowledge before the workshop, which is lower than the previous mild to moderate comfort level reported by various groups of ATs¹³ but similar to those of athletic training students.¹¹ After the workshop, confidence increased to a moderately confident level exceeding that of confidence reported in previous athletic training research.¹³ This increase in confidence correlates positively to the gain in EBP knowledge and may, in theory, translate to overall improved confidence in the process of EBP, which is an important factor in support of implementation and potential transference to practice. Athletic trainers having moderate to extreme confidence in the EBP process could, in the future, transition to expanded use of EBP in clinical practice and better understanding of the clinical EBP programming approved by the BOC.

Evidence-Based Practice Resources and Barriers

Transference of EBP knowledge to practice is not without barriers. Participants in this study specifically identified perceived barriers in the areas of knowledge related to EBP, time, accessibility of evidence, and the culture of their clinical practice environment. These results are consistent with barriers cited within athletic training^16–20,27 as well as other health care professions.^28–33 Although these barriers are not novel, thought and reflection should be given to if and when we can anticipate changes to these barriers, particularly now that all ATs are required to obtain 10 EBP CEUs every 2 years. Continual improvement of EBP knowledge and related confidence may help to decrease ATs' perceived barriers to implementation. Although our study did not show a change in barriers despite an increase in perceived knowledge, it will be important to assess the long-term effects of CPE interventions to better understand how EBP implementation changes.

Lack of identified resources (ie, knowing where to seek evidence) is a common barrier to incorporation of EBP.^{17,18,20,21,35} The acknowledged lack of accessibility, availability, and applicability of evidence is also supportive of previous findings.^16–21 Participants within this study addressed resources within both the barriers question and the envisioned-use question, thus suggesting that resources are a barrier to envisioned use of EBP. Although the short 5-hour course did not demonstrate change in specific resources used by ATs, participants did indicate that as a result of the workshop they learned how to better access and streamline information in a manner that is easier to process. Future CPE opportunities should not lose sight of the fact that many clinicians do not fully understand how to best access information and process it in a manageable way.

Whereas many clinicians work independently in their setting, identifying a network of professionals to assist in accessing information may help to ease access to resources as well.³⁶ For those clinicians who serve as preceptors, it may also be possible to gain information from the athletic training program they are affiliated with. Regarding applicability of evidence, the information disseminated, both scholarly research and case reports, should continue to expand. Specifically, evidence targeting clinically based questions, perhaps on topics suggested by practicing clinicians, should help to improve the body of applicable evidence.^17,35

In further discussion of resources, a portion of this workshop was designated to identification of resources varying from free access to paid subscription and to ranking the preferred sources for information. It is interesting to note that less than 5% of responses for databases and resources used to identify evidence included the Cochrane Database of Systematic Reviews, which is considered to house the highest level of clinical evidence. Whereas the Cochrane database presents the highest quality of evidence, the availability of this database outside of the university or medical setting is extremely limited, which could also lead to its lack of use.

It is somewhat disconcerting that the barriers addressed in this study are the same as those identified for the past 5 years in athletic training despite an increased emphasis on EBP in the profession.^16–20,27 The availability of specific EBP programming, including though not limited to online modules, condensed literature reviews in the forms of critically appraised topics, and clinical bottom lines, has grown exponentially. Whereas these resources are helpful, it is up to individual ATs to actively work to overcome the barriers applicable to their situation. Transitioning to a culture of EBP will come with constant repetition and exposure.³⁵ In order for athletic training as a profession to be competent and inclusive in all knowledge areas identified by the National Academy of Medicine, formerly the Institute of Medicine,³ ATs have a professional responsibility to learn and integrate EBP into practice to align with other health care professions. Learning theories indicate that CPE and postgraduate education decisions tend to be driven by self-motivation and clinical practice.³¹ One of the biggest factors related to the commonly cited barriers may be a lack of motivation to learn and truly understand the concepts of EBP. Athletic trainers need to see EBP not as a required item to check off their certification maintenance list, but instead as a necessary method of acquiring and retaining information for transference into professional practice.³⁷

Envisioned Use of EBP

Our findings regarding envisioned use of EBP are unique to previous research in the AT population. The preworkshop qualitative data were insightful regarding envisioned use but also much less specific than postworkshop responses, which related more to the EBP process and concepts presented in the workshop. These findings posit that instruction on foundational concepts of EBP and improved perceived knowledge, as demonstrated in this study, may influence how ATs envision use of EBP concepts. For example, postworkshop, participants indicated that they envisioned using PROMs during patient care to track progress and outcomes of their patients; there was no reference to patient-reported outcomes by participants in the preworkshop qualitative data. As a component of improving individualized patient care, PROMs are a useful tool in the process of EBP.³⁸ These outcome measures aim to capture whether health care services provided by a clinician improve a patient's health as related to his or her impairment, goals, and experience. Emphasis on patient-centered outcomes has increased as effectiveness of care is now being evaluated from both the clinician and patient viewpoints.³⁷ In addition, patient-centered care is another competency identified by the National Academy of Medicine³ that all health care providers should possess. In 2008, Valovich McLeod et al³⁸ presented information intended to enable evidence-based AT clinical practice through use of PROMs. Specifically, this article identified criteria to evaluate patient-based outcome assessment tools for potential use in clinical practice. Despite the publication of this information, ATs are still not universally familiar with patient-reported outcomes and therefore, are not commonly incorporating PROMs into AT clinical practice.³⁹ A potential theory for this lack of use may be attributed to challenges related to collection of this type of information including time and role strain.

Although teaching EBP was an identified envisioned-use category by some participants in this study, evidence supporting best practices for teaching CPE,³⁷ particularly in relation to EBP,^20,35 is lacking. Whereas it is recommended that ATs serving in educator and/or preceptor roles not only instruct EBP concepts but also be able to model these skills,²⁰ it is unknown how often these techniques are occurring and what the resulting impact on patient care is, if any. Interacting with athletic training students, for example, serves as an opportunity to encourage, model, and direct the incorporation of the tenets of EBP into clinical practice.²⁰ Our participants indicated a desire to contribute to educational opportunities in this manner. However, in order for application of EBP within clinical practice to be successful, ATs must have the foundational knowledge to provide correct instruction and feedback.

The content of this workshop resulted in an increase in perceived EBP knowledge and confidence in that knowledge, which supports previous researchers' recommendations that the success of teaching EBP through CPE and educational interventions be evaluated.^35,37 These increases in knowledge and confidence may translate to improved educational opportunities and modeling of EBP in future interactions with AT students. In addition to these interactions with students, it is plausible that educational opportunities with patients, coaches, administrators, and so forth could be improved through a better understanding of foundations of EBP so ATs can more effectively disseminate information and justify care to stakeholders who request such information.

Limitations

This study is not without limitations. One such limitation exists in the convenience sample used. It is possible that ATs with a self-perceived lower-level knowledge of EBP concepts may have registered for this course. The self-report nature of the survey instrument also could be a limitation because it assumes that participants answered honestly and to the best of their ability. In addition, the EBCKAU survey was created to evaluate knowledge of foundational concepts of EBP only, not advanced or clinically based EBP concepts. Also, the short 5-hour structure of the course makes these results generalizable only to those enrolled in brief courses targeting foundational knowledge of EBP and assessed only immediate perceived knowledge and not long-term changes in knowledge of EBP concepts.

An additional limitation could be the fact that the number of presenters varied between workshop locations. The research team attempted to minimize this limitation by conducting the workshop with all 3 presenters before the 1-presenter format. The lack of statistical significance between locations in regard to preworkshop and postworkshop knowledge scores suggest this instructional change had little effect on the knowledge outcomes of the workshop.