Using a Critical Appraisal Assignment to Infuse Evidence-Based Practice into a Therapeutic Modality Course
It is the responsibility of athletic training educators, through curriculum and clinical experiences, to engage students towards adopting evidence-based practice (EBP) into their practice. The initial task of implementing EBP into a curriculum or course can seem like a large task for educators and students. As a way to start scaffolding EBP concepts across the curriculum, a modified critical appraisal assignment was developed to teach therapeutic modality concepts. The purpose of this action research project was to demonstrate how a modified critical appraisal assignment can be used to introduce the process and aspects of critical appraisal and begin scaffolding the development of critical appraisal skills over time. The objectives of this study were to evaluate the students' ability to (1) successfully locate relevant research needed to answer clinical questions and (2) successfully appraise the literature according to basic EBP strategy. From a program perspective, the modified critical appraisal assignment was a starting point from which to include EBP principles into didactic coursework. Seventeen athletic training students completed 3 modified critical appraisal assignments pertaining to the use of therapeutic modalities. Each paper included 5 sections: (1) clinical question, (2) key clinical findings, (3) clinical applicability based on information from the appraisal and significance of results, (4) article comparison table, and (5) implications for clinical practice, patient education, and future research. The instructor evaluated the assignments blind. Students were generally able to complete the critical appraisal assignment; however, students had difficulty locating research that answered the clinical question. Students struggled to relate the key clinical findings of the research articles and implications for clinical practice to the given clinical question. Findings from this study have informed faculty teaching, including introducing EBP skills earlier in the curriculum and inserting assignments that stress various aspects of the critical appraisal process.Context
Objective
Design
Conclusions
INTRODUCTION
Evidence-based practice (EBP) is the integration of best research evidence with clinical expertise and patient values.1 Teaching EBP is an essential component of a health profession's curriculum. If educators and preceptors are not teaching and applying the process of EBP, students will be unable to sustain their practice through current evidence.2–6 Students within an athletic training (AT) environment should be good consumers of available literature. At minimum, they should be able to generate answerable clinical questions and locate relevant literature that has already been synthesized for clinical use. In cases where synthesized literature is unavailable, they should be able to locate the best literature available, apply rapid critical appraisal techniques, reflect on their findings to determine if the information should change their practice, and continue the cycle.7,8
Critically appraising the research is used to determine if study results should be translated and applied to clinical practice.9 Once the study is appraised, the clinician should be able to determine if its results are reliable and valid, as well as if the findings answer the clinical question.9,10 Critical appraisal is a skill best practiced and learned over time. Clinicians skilled at the clinical appraisal process have both clinical perspective and experience as well the ability to identify clinical questions, locate literature, assess validity and reliability of a particular piece of literature, and determine the clinical significance of the findings applied to practice. Students beginning a professional education program lack the knowledge, skill, and clinical perspective of the professional; therefore, they lack the knowledge and skill needed to critically appraise the literature.3 Educators are challenged with introducing professional content with students at the same time as the critical appraisal process. In order to facilitate learning over time without overwhelming students, a planned approach to introduce sequential learning of both professional content and EBP should be developed.3 If planned properly, the critical appraisal process can be used as an active learning strategy to engage students regarding current literature in the field.
The purpose of this project was to demonstrate how a critical appraisal assignment, included as part of a therapeutic modalities course, can be used to describe the students' ability to learn the critical appraisal process over the course of the semester. The assignment was developed using the critically appraised topic (CAT) as the framework; however, it was modified to scaffold student learning of the critical appraisal process because this was the first time students were exposed to both the content of therapeutic modalities and the critical appraisal process. The objectives of this study were to evaluate the students' ability to (1) successfully locate relevant research needed to answer clinical questions and (2) successfully appraise the literature according to basic EBP strategy. From a program perspective, the critical appraisal assignment was a starting point from which to include EBP principles into didactic coursework. An understanding of EBP and best-practice pedagogies is necessary to be successful in educating students and to disseminate best practices to other professionals.
METHODS
Participants
Seventeen AT students enrolled in a dual-listed therapeutic modalities course participated in this study. Both undergraduate and entry-level graduate students at the institution enrolled in this course. Fourteen students were part of the undergraduate AT program and 3 students were part of the entry-level master's AT program. All 17 students completed the therapeutic modalities course within the second semester (spring semester) of a 4-semester AT program. Students had prior knowledge of lower extremity assessment skills and acute care assessment and skills.
Overview of the Therapeutic Modalities Course
As an introduction to EBP, a critical appraisal assignment was included within the formative assessment structure of the course. This was the first semester the instructor had taught the course and the concept of EBP was not part of the previous course design. Students had no prior knowledge or skill in EBP; however, they had participated in a library research activity during their first semester of the AT program. The focus of the library research activity was to teach students how to access library information and perform online searches in medical/AT relevant databases. These databases included, but were not limited to, PubMed, CINAHL, EBSCOhost, and MEDLINE. Because students had been exposed to basic strategies to locate resources, this was not repeated at the beginning of the therapeutic modalities course.
For each therapeutic modalities topic area addressed in the course, the following pedagogical steps were followed. First, students were taught the basic theory and application of the therapeutic modality. Then, subsequent class meetings were used to discuss topic-specific research articles provided by the instructor. The class discussions provided an avenue for students to understand how to read and appraise the journal articles. The in-class learning activities included using a critical appraisal checklist developed by the Centre for Evidence-Based Medicine11 to help determine level of evidence and strength of recommendation. The above steps were used throughout the semester with course materials. Students were taught the knowledge and skills related to the therapeutic modality before they completed the critical appraisal assignment. Students were given 4 weeks to complete and submit the assignment.
Clinical Questions and Specific Requirements of the Critical Appraisal Assignment
Because of student inexperience in the use of therapeutic modalities, as well as EBP, and limited class meeting days, the instructor developed the clinical question for each of the 3 topic areas (Table 1). Over the course of the semester, students were given the 3 following clinical questions: (1) To gain desired physiological effects, what is the appropriate length of an ice pack/cryotherapy application? (2) What is the efficacy of utilizing therapeutic ultrasound to positively promote joint range of motion? and (3) When using forms of electrical stimulation for decreasing pain, how does electrode placement affect the desired outcome? Because of the need for students to learn overall therapeutic modality concepts, the clinical questions did not follow the patient, intervention comparison, outcome (PICO) format.9 Students were learning concepts of therapeutic modalities at the same time they were locating resources; therefore, the clinical questions were developed to allow students to locate meaningful studies to answer the questions. The instructor purposefully kept the clinical questions broad to allow for student learning of the overall concept.
Once a clinical question was introduced, students were required to locate 3 quality articles that related to each of the clinical questions. The composition of the critical appraisal assignment included a written expression of the critical appraisal process. In the critical appraisal assignment, students included a bullet list of the key clinical findings; a paragraph regarding clinical applicability based on information from the appraisal and significance of results; a comparison table for the 3 journal articles that included study design, number and type of participants, intervention investigated/methods–experimental and control interventions, the main findings (results), and conclusion; and a narrative that addressed implications for clinical practice, patient education, and future applicable research. Specifically in this section, students included how the appraised literature would affect their clinical practice and patient education. As well, students also stated if additional research was necessary to help make an informed decision about the use of the therapeutic modality in relation to the clinical question.
Assessment
The instructor designed a scoring rubric to assess the students' ability to synthesize and appraise each article. The scoring rubric was given to the students before they submitted the first assignment. The rubric outlined specific details that needed to be included within each section of the appraisal for full points to be awarded. Table 2 outlines the scoring rubric used to grade the assignments. The instructor was blinded to who submitted each assignment to improve intrarater reliability during the grading process.
The scoring rubric addressed the key content areas of critical appraisal and information synthesis. Sixty percent of the critical appraisal assignment related to the critical appraisal process (10 points each for key clinical findings; clinical applicability; and implications for clinical practice, patient education, and future research). Twenty points were delineated for the student's ability to describe each study in a table format.
Students completed 3 separate critical appraisal assignments through the 16-week semester. Students were encouraged to seek out additional help if they were unsure about a component of the article or assignment. Scoring rubrics were completed and returned to the students before the submission deadline for the subsequent assignment.
RESULTS
Statistical analyses were performed using SPSS version 18 (IBM Corporation, Armonk, NY). Descriptive statistics, specifically means and standard deviations, were used to determine average scores for each section of the critical appraisal assignment. These results are illustrated in Table 3.
Descriptive statistics revealed that students averaged 8.41 points out of 10 on the clinical findings section, 8.57 points out of 10 on the clinical applicability section, 19.22 points out of 20 on the table information, and 8.43 points out of 10 on the implications for practice, education, and future research section. Students averaged .06 points out of 2 for extra credit related to grammar and syntax.
DISCUSSION
The purpose of this project was to demonstrate how a critical appraisal assignment, included as part of a therapeutic modalities course, can be used to scaffold a student's ability to learn the process of critical appraisal over time. The objectives of this study were to evaluate the students' ability to (1) successfully locate relevant research needed to answer clinical questions and (2) successfully appraise the literature according to basic EBP strategy. From a program perspective, the critical appraisal assignment was a starting point from which to include EBP principles into didactic coursework.
Effective critical appraisal by health care professionals requires clinical knowledge and expertise in order to frame the process, but also an understanding of the technique of critical appraisal. Adding components of the critical appraisal process early in an AT curriculum allows students to practice techniques of critical appraisal as well as actively engage in the learning of core knowledge. This assignment was not meant as an outcome project, but rather a beginning and less daunting approach to EBP.
Implications for Educators
To help facilitate critical thinking and learning of EBP concepts, AT program faculty (didactic and clinical) need to incorporate learning activities that facilitate the adoption of EBP principles.12–14 One approach is through the critical appraisal of research. Once a clinical question has been developed, students need to be able to access current literature, determine the relevance of the literature to the clinical question, and assess the research design and level of evidence.9 Students come to our AT program with different skills and intellectual abilities. By incorporating this assignment, AT faculty at our institution were able to understand where students were proficient and where they needed more help and guidance. Students in this study had a difficult time relating the key clinical findings of research articles to the given clinical question developed for the assignments. In part, this may have been due, fundamentally, to the inability of the students to adequately locate research. If the articles selected were not related to the clinical question, it is difficult to make any reasonable determination of clinical relevance. Difficulty in locating relevant sources is similar to findings from research in nursing that indicate students struggle with the EBP process early in their academic careers because of their inability to identify, access, and evaluate research.15 Findings from this nursing education study also suggested high self-reported confidence levels in students with regards to accessing library information; however, this was not confirmed. The majority of the 17 students from this nursing study stated they were aware of search engines and databases to access peer-reviewed journal articles15; however, being aware of and actually searching, identifying, and selecting appropriate articles are very different skills. The top 3 databases for nurses and other health professionals include CINAHL, MEDLINE, and PsycINFO.16 The students of this current study had access to each of these databases through the campus library.
It is also possible that the inability of the students to locate research could be related to the limited amount of quality research related to therapeutic modalities. In addition, subject matter of the critical appraisal assignments may have affected the students' ability to find relevant articles. The first critical appraisal assignment related to cryotherapy, the second assignment related to ultrasound, and the third assignment focused on electrical stimulation. The types of electrical stimulation are numerous, and therefore the wide range of search terms may have affected search results. Although students were familiar with the material before submitting the assignment, this issue still may have affected their ability to find relevant research.
It is one thing to access correct databases, but it is another thing to use correct search terms. Students were not required to list the search terms they used. Considering there are multiple descriptors, terms, and conditions to use with these searches, students could have struggled with determining which search terms to use to produce the desired results. Future assignments will require students to list databases and search terms used in their literature searches. This will help faculty understand how students are searching databases so they may provide feedback to students in order to improve the effectiveness of the students' search processes.
Student scores related to key clinical findings decreased over the 3 assignments. Students were able to list key clinical findings from each article; however, the goal was to synthesize information from all 3 articles. Students struggled to develop holistic clinical findings related to the clinical question. Appraising research can be a difficult task to understand.6 Comprehending statistical measures and results can help appraisal methods.10 Students were given limited instruction relative to statistics as part of this project. Throughout the semester, in-class learning activities were used to promote the comprehension of statistical methods used in research; however, it was determined that students may need a more extensive approach. With future assignments, students will submit their critical appraisal checklists and Physiotherapy Evidence Database (PEDro) scoring with their assignments. This will provide the instructor a way to assess a student's clinical reasoning to determine if the student understands the research article's methodology and results. Also, students may benefit from taking an undergraduate or graduate statistics/research methods or epidemiology course to help them understand statistical measures in greater detail.
Implications for AT Programs
Clinical Experience
Results of this study also support the students' inability to develop implications for clinical practice, patient education, and future research. Students unable to determine key clinical findings would also find it difficult to relate how the information affects their clinical practice. Lack of clinical experience and/or practical application of the therapeutic modality technique may contribute to this phenomenon. Students enrolled in this course had completed only two 8-week clinical experience rotations. Although they may have seen a certain therapeutic modality used, they may not have fully understood the theory behind the application in relation to the pathology and injury repair process. Researchers state that clinicians have difficulty with EBP practices if the applicability of the subject matter does not fit into their clinical practice.17 If students were unable to understand the applicability of the therapeutic modality in clinical practice, it would be difficult to develop and understand how the given therapeutic modality would affect patient outcomes.
Practicing clinicians believe time, availability of databases, and lack of understanding of research methodology make the EBP process challenging to adopt.12,18,19 According to Manspeaker et al,12 a barrier to the use of EBP can be a lack of applicable resources. Considering this problem, educators need to provide students with appraisal strategies that can be easily implemented in the work environment to support creating a culture of EBP within the organization.19
Didactic Coursework
Students within an AT environment should be able to consume literature that has already been synthesized, formulate sound clinical questions, locate literature relevant to a specific question, critically appraise literature, reflect and determine if the literature affects practice, and continue the cycle.3 This critical appraisal assignment was used to start embedding EBP throughout our entire AT curriculum. Ciliska3 stated, “What is required is not one class but an evidence based curriculum where skills development and expectations of use of evidence are embedded across courses and across years in the academic and clinical settings.” In much the same way, physical therapy students have been taught to critique the research literature and develop the necessary skills for EBP.20 Results and conclusions of the assignments have been used by our AT program to implement EBP principles within the curriculum. According to Denegar and Hertel,13 critical review is necessary to establish practice guidelines and this in turn will define educational preparation and clinical practice. To do so in our AT program within the first semester, students now have an introduction to EBP principles and the critical appraisal process (Table 4). Throughout subsequent semesters, formative assessments are embedded to continue scaffolding the students' learning and adoption of EBP concepts (Table 5). In the third semester of the AT program, students now complete a full CAT following journal submission guidelines (Table 6). The above changes were made specifically to our AT program; however, any AT program needs to understand “that teaching research and statistics courses is not enough”3 to enable students to successfully use EBP principles. The ideal curriculum integrates EBP across all levels and courses.3 A starting point for programs is to determine expectations of student skills and performance at various levels, then develop relevant assignments that reflect those expectations.3
CONCLUSIONS
Using a critical appraisal assignment can be a positive active learning strategy for AT students to critically appraise research and begin to incorporate EBP into their clinical practice. In using this teaching strategy, educators must, over time, teach students the steps of critical appraisal and continue to enhance their research skills through active learning pedagogies. Components of EBP should be included throughout an AT program so that students are able to see how research will affect their clinical practice. As this was a starting point for the AT curriculum, our faculty have developed a framework from which to include EBP principles from the beginning to the completion of the AT program. Learning EBP principles over time will help students understand the 3 EBP components and how they work together to improve patient outcomes.
Contributor Notes
Dr Zwart is currently an assistant professor in the Department of Health and Nutritional Sciences at South Dakota State University.