A Tiered Approach for Integrating Patient Outcome Measures into the Curriculum
Knowledge and understanding of effective practices for integration of patient-reported outcome measures (PROMs) within athletic training curricula are necessary to bridge the gaps between didactic application, content assessment, and clinical implementation. To provide athletic training educators with a tiered approach to implement content and assessments related to PROMs in the athletic training curriculum. An emphasis in the athletic training community is the need to incorporate patient-oriented evidence that matters into clinical practice. One way of achieving this transfer of knowledge is incorporating PROMs into athletic training curriculum. The 2020 Commission on Accreditation of Athletic Training Education (CAATE) Standards include PROMs and strategies to evaluate them for use in clinical practice to improve patient care. Thus, stakeholders responsible for students' education must have the knowledge and ability to properly address these standards in order for students to utilize these skills as future athletic trainers. This article offers an approach for educators to teach and assess PROMs within their athletic training curriculum. By utilizing an effective teaching approach for the implementation of PROMs, educators, preceptors, and students may collectively integrate these validated tools accurately into patient care to provide a more holistic practice. In addition, using a tiered approach will increase understanding and confidence for athletic trainers who have identified barriers and may not have prior clinical experience in the implementation of PROMs with patient care. The knowledge and use of PROMs are expected of students in CAATE-accredited athletic training programs. In order to ensure and enhance the transfer of knowledge from the didactic setting to clinical practice, the use of a tiered approach may benefit athletic training educators as they integrate this content into coursework. In turn, perhaps future clinicians may be more apt to value the benefits of PROMs.Context
Objective
Background
Description
Clinical Advantages
Conclusions
KEY POINTS
-
CAATE 2020 Standards for Accreditation of Professional Athletic Training Programs, specifically Standards 60, 62, and 69, are related to the inclusion of the International Classification of Functioning, Disability, and Health Model and evidence-based practice, lending to the use of PROMs to drive the treatment and rehabilitation of patients.
-
The expectation is that content knowledge and skills related to PROMs are obtained by athletic training students through didactic and clinical education. The 4-tiered approach can help educators and preceptors become more familiar with the use and benefits of PROMs, which may coincidentally aide in teaching this content to athletic training students.
-
The use of a tiered approach supported by targeted instructional strategies throughout the athletic training curriculum may encourage athletic training students to implement PROMs during patient care as future athletic trainers.
INTRODUCTION
Standardized assessments in clinical practice exist to obtain critical knowledge relative to a patient's status that should be used to assist with injury or illness diagnosis but that also provides valuable information on the effectiveness of treatment.1 Assessments should include disease-oriented outcomes along with “patient-oriented evidence that matters” (POEM) to ensure the health care provider is considering the patient's perspective in a patient-centered care model.2 Measures have been developed that, when properly delivered, can provide the clinician with a greater understanding of the patient's condition. These patient-reported outcome measures (PROMs) then translate the patient's perspective into data to be interpreted by the clinician to provide a meaningful whole-patient picture. Regardless of the documented benefits of incorporating PROMs, which include increased communication between the health care provider and patient,3,4 fostering a shared responsibility in injury management,1 patient satisfaction with care,3,4 and better treatment outcomes,5 authors1,6 have noted that clinicians may be reluctant to utilize PROMs because of the perceived barriers they present.
Within the athletic training profession, a lack of inclusion of PROMs in patient care has been well documented.5 Barriers to PROM usage in clinical practice include a lack of training in assessment and implementation, lack of comfort with technical aspects (instrument essentials), and difficulty associated with interpreting the PROMs.5 It is unclear if these documented barriers affect athletic training educators in their dissemination of course content to students, as educators rely on their clinical competence and experiences in patient care.7
Effective teaching relies on the educator possessing both pedagogical and content knowledge, referred to as “pedagogical content knowledge,”8 in order for students to learn and promote the transfer of cognitive and psychomotor skills. A teacher's knowledge is reflected in how she plans, selects, and sequences the subject matter.9 On one hand, there may be athletic training educators who have knowledge of PROMs but are not able to direct or transfer that knowledge10 to athletic training students. Having that pedagogical knowledge allows for recognition of the students' learning abilities, teaching through differentiated instruction, and making adjustments when needed, all of which may not occur with athletic training educators.10 On the other hand, athletic training educators may lack content knowledge in the area of PROMs and have identified barriers in the use of PROMs, similar to those barriers noted by clinicians. Regardless, dissemination of PROMs content to students must occur, especially because the Commission on Accreditation of Athletic Training Education (CAATE) has identified standards relative to the inclusion of PROMs in the education curriculum. While researchers1–6 continue to educate athletic trainers in the benefits of PROMs, athletic training educators as well as preceptors may benefit from instructional support through a tiered approach in order to provide athletic training students with a clear understanding of how to best implement POEM, specifically PROMs, during their didactic and clinical education.
The remainder of this article describes a 4-part integration of PROMs and related subject matter into an athletic training curriculum. This tiered approach includes (1) teaching discrete content, (2) teaching application of content, (3) an assignment that incorporates live patient care, and (4) an assessment focused on student reflection of the assignment. This approach was built upon the development of instructional strategies surrounding domains of learning, teaching styles, and problem-based learning. In order to accomplish PROMs-related learning objectives, the authors recommend incorporating the tiered approach across 2 semesters to ensure students have ample authentic patient encounters to allow for reflection on these experiences. Navigation through the outlined tiered approach will be supported by pedagogical knowledge with the intent of helping athletic training educators to improve their dissemination of content, lending to increased student value in the integration and use of PROMs with patient care.
Instructor Preparation
In order to promote successful completion of all tier outcomes, the educator must first recognize the information that exists relative to this subject area, determine how to disseminate the content, and evaluate if the students are able to relay instrument essentials and clinical utility. First and foremost, the educators should consider improving their own knowledge of the subject matter through structured learning,11,12 to include a review of existing literature, attendance at continuing education programs, and developing an acquaintance with PROMs, whether during a real patient encounter or through a patient scenario. Structured learning on the part of the educator will lend to supporting student achievement of subject matter learning outcomes.11,12 This learning must continue over time to enhance the educators' depth of knowledge,10 comfort with the content, and clarity regarding the benefits of PROMs. Then educators must apply their pedagogical knowledge to select appropriate strategies to foster superior teaching that promotes positive, student-centered learning13 and maximizes engagement.14 Once athletic training educators are acquainted with the subject matter, they might consider infusing the following tiered approach into their curriculum.
Tier 1: Exposure to Theoretical Components
The suggested avenue through which to disseminate content on PROMs for the first time is the “Command Teaching Style”9 and discussion9,15 strategies. With the Command Teaching Style, the educator makes all the decisions, and the students must respond based on the information provided. The benefits to inclusion of the Command Style are that it is very teacher focused and the assumption is that the student's level of knowledge and use of the content is low to almost none. Students accurately learn the content in a short period of time and develop automaticity in student responses to the subject matter.9 Physical and cognitive involvement have been found to be present during use of the Command Teaching Style; in addition, preference is given to this style when the student is first learning a skill or content, as opposed to use of a student paired practice situation.16
During the first tier, students should be able to recognize the criteria for selecting PROMs and identify the development process and validation of a PROM instrument. Background knowledge on instrument essential items such as instrument development, reliability, validity, responsiveness, interpretability, and precision17,18 should be introduced by the educator when PROMs are initially taught. These elements can be useful in establishing whether or not the instrument is appropriate for identifying POEM. In addition, knowledge of clinical utility measures, such as acceptability, feasibility, and appropriateness, is important when making decisions about the application of PROMs.17,18 A recommended resource to accompany the theoretical components is the article by Lam et al18; it is a great tool with which to enhance comprehension of PROMs essential elements and clinical utility. A concise summary of clinically relevant PROMs, which was previously published by Lam et al18 (Table 1), is used as a quick reference guide and the basis for the 3 categories of PROMs (region specific, generic, and single item) students will be using throughout this tiered approach. The authors have found this content18 useful in alleviating the overwhelming feeling that every PROM instrument must be discussed with students.
After dissemination of the introductory content on PROMs, the authors suggest increasing students' critical thought on the content through peer discussion.15 This discussion begins with only the single-item PROMs. Paired students are asked to first review their assigned PROM and then respond to instructor-set questions on applicability, responsiveness, and strengths and weaknesses of the instrument (Table 2). The authors recommend starting with single-item instruments like the Numeric Pain Rating Scale or the Global Rate of Change because they are the easiest for students to interpret and are frequently used in clinical practice.6 Another advantage of a single-item instrument is that the time required for completion is minimal. The goal of the peer discussion is to reinforce PROM selection criteria and recall essential elements and clinical utility of the instrument. The single-item instrument can also be included in a case-based scenario in which the students are expected to provide an explanation of the instrument and its purpose in clinical practice. When developing discussion questions, whether for a large group or a pairing of students, in order to draw upon their cognitive process of memory recall and discovery the educator should avoid ambiguous words in the questions.9
The educator should not simply ask the students to discuss their thoughts on the PROM, as this may lead to misunderstanding and uncertainty9 in their reactions to the PROM. In addition, by incorporating the discussion activity, the dissemination of content has moved from teacher centered to student centered, thus forcing the students to employ critical thought15 even during an introductory lesson with PROMs. The Guided Discovery Teaching Style, in which the teacher develops a logical set of questions in order for the student to discover the determined response, aligns with the promotion of critical thought. Within Tier 1, the student is not necessarily reproducing previously taught content, but rather is producing content that is interconnected to what was already taught.9
Once students fully explain the utility of a single-item instrument, a discussion of the generic outcome instrument should ensue. Generic outcome instruments are beneficial for a wide range of patient populations and offer a broad perspective on health-related quality of life (HRQoL).2,19 The generic outcome instrument we recommend is the Disablement in the Physically Active (DPA) scale because it was specifically designed for use in the athletic population.20 The DPA scale is a 16-item instrument that captures a broad range of HRQoL dimensions, including patient dysfunction, skill performance, and patient well-being.20 Similar to the single-item instrument discussion, introduction of the generic outcome instruments begin with a peer-to-peer discussion. Within their peer groups, students should complete the DPA independently. The goal of this exercise is to help students become familiar with the instrument items and to provide a thorough explanation for how and when to use the instrument in clinical practice. Students can also engage in conversation with clinical preceptors regarding the inclusion of generic instruments at their clinical sites to collect HRQoL data.
When the student is able to gain a sense of the broad and multidimensional components surrounding the patient's complaint, the educator can transition the student's focus to region-specific outcomes. Region-specific outcome measures have specific advantages for obtaining relevant patient information, but it can be difficult for students to determine the best option to use, given the large number of available instruments and variation in the information gathered. For example, if a patient reports to the athletic training clinic complaining of ankle pain, the Foot and Ankle Disability Index21 and the Foot and Ankle Ability Measure22 would both provide region-specific information, but it might be difficult to discern which one is better based on the patient's current condition.
Both outcome measures provide information regarding the specific body region and will produce a score of level of ankle dysfunction.2,19 Region-specific outcome measures are advantageous in that they are sensitive to changes over time of the specific dysfunctional anatomic region2,19; however, many region-specific instruments neglect the broader patient profile or HRQoL data.2 As students review various region-specific outcome measures and engage in peer-to-peer discussions, attention should be placed on what information the instrument items are attempting to obtain and how this information will affect treatment strategies. Additionally, since region-specific outcome measures are typically confined to the anatomical region of dysfunction, how will students obtain information on more global patient health status as well as HRQoL? Students are again provided discussion questions (Table 2) to facilitate the learning activity.
By introducing these essential elements, students should be able to generate their own interpretation of any PROM. The guided peer discussion (Table 2) activities allow students to consider the foundational knowledge in a controlled environment, and the educator is able to intervene by asking questions to assist in stimulating critical thinking.23 When the learning outcomes have been achieved, the educator can shift focus to the student's selection of PROMs and application during a case-based simulation, which adds a new layer of learning.
Tier 2: Application of Case-Based Simulation
Problem-based learning is a well-documented instructional method in the athletic training literature24–31 that supports student learning; the educator is able to present realistic patient scenarios for students to apply their knowledge in a holistic approach.32 The authors integrate a higher level of thinking through the use of case-based simulations (CBSs) during a mock patient and clinician (athletic training student) experience to enhance student engagement and problem-based learning during the second tier of integrating PROMs into the curriculum. A CBS has been described by Walker et al30 as a preplanned clinical scenario carried out by a peer student who has not undergone any formal training to depict the injury. This progression supports the need for students to develop clinical competence during realistic situations with CAATE Standards 60, 62, and 6933 through the use of positively documented instructional methods.34,35 Therefore, the Tier 2 learning outcomes, as outlined in Table 3, require students to recognize PROMs distribution, provide context for inclusion in clinical practice, and support clinical judgement in the treatment and rehabilitation of injuries.
In Tier 2 it is recommended that each athletic training student be assigned a preplanned CBS (eg, ankle pain, shoulder pain) and required to identify at least 2 PROMs for distribution during the CBS. The scenario given to each student is that the injury has occurred at the end of practice and a reevaluation will occur the following day, which allows students time to select and research the PROMs they would like to utilize during the CBS experience. Students use the aforementioned content18 in Table 1 as a framework for PROM selection, which must include both a generic and region-specific outcome measure. Students must return the following class period with all of the following: a brief explanation for the patient as to why they will be using the selected PROM, justification for selection of each PROM, and a plan for distribution (ie, frequency with which the PROM will be distributed).
During the succeeding class period, students are placed in groups of 2, in which one student portrays the role of the mock patient and one student serves as the mock clinician. The mock patient is given a written template and debriefed by the educator for 3 to 5 minutes30 on information directly relevant to the case (Table 3). The mock patient is expected to rely on the information provided in the written template and their previous content knowledge to answer questions during the medical history and to complete the PROM instruments. During the CBS, the mock clinician obtains a thorough medical history and is required to administer the 2 previously gathered PROM instruments along with a brief explanation for why these instruments were selected. Immediately upon completion of the CBS, the mock clinician answers a series of questions related to the barriers to implementation of PROMs in clinical practice (Table 3). Students should recognize the length of time of PROM completion, frequently asked questions, questions that raise concern or may be misinterpreted, as well as how the information will impact the treatment and rehabilitation plan. While relying on their personal experiences with patient evaluation as well as their content knowledge on PROMs, educators should facilitate a discussion about the mock patient and mock clinician experiences after the exercise to ensure a thorough understanding of advantages, disadvantages, and strategies for each PROM integration.
Depending on when Tier 2 is implemented within an athletic training program's curriculum, students may have difficulty using the PROM to enhance rehabilitation and may require additional mentoring to ensure that each PROM is used effectively. Once the educator believes the learning outcomes have been achieved, Tier 3 may be implemented. The authors have completed Tiers 1 and 2 in a didactic evaluation course and Tiers 3 and 4 in a clinical course that is sequenced after that evaluation course. Therefore, the educator should also consider the curriculum sequencing and course learning outcomes to decide where integration of each tier would be most effective.
Tier 3: The Authentic Experience
The purpose of Tier 3 is to have the student integrate PROMs during a real-time patient encounter, which encourages the students' transfer of knowledge to clinical practice.34,35 Students value authentic experiences, as identified by Mensch and Ennis,36 and therefore the authors feel an authentic encounter is critical to embed in the curriculum not only because it raises the level of cognitive function with the subject matter but also because it encourages the students to appreciate the transfer of standards to clinical practice. The recommended Tier 3 assignment, as outlined in Table 4, allows for interaction with the preceptor, which in turn may provide an opportunity for reciprocal learning to occur.37,38 As previously discussed, superior teaching lends to student learning17; therefore, the preceptor should not feel “siloed,” as the outcome in this tier is to bridge the gap between didactic and clinical education. The preceptor may lack knowledge in the area of PROMs or may have difficulty educating and assessing the students' implementation of the instrument during patient care. To accommodate various levels of content and teacher expertise,11 it is important for the educator to maintain lines of communication with the preceptor to clarify and reinforce theoretical concepts as well as the skills related to dissemination and interpretation of PROMs. Additionally, educators may share structured learning11,12 opportunities related to this content so preceptors are able to gain subject matter knowledge. Given the proposed outcomes of this portion of the assignment, students should be permitted the appropriate amount of time to integrate the PROM in clinical practice in order to fully appreciate the effectiveness of the PROM on patient assessment findings and to use the findings to develop a treatment plan. Therefore, the authors allow the student the entire semester to implement PROMs, multiple times, with the same patient. This encourages the preferred real-time patient encounter30 while accommodating the timetable of the preceptor.39
Students are instructed to implement a global and region-specific PROM based on the circumstances surrounding the patient encounter to ensure they are capturing the full extent of patient dysfunction. Students are encouraged to recall the content previously taught in Tier 1 and applied during the CBS in Tier 2. Instructional support materials, such as the previously used content18 in Table 1, are provided to assist the student, and potentially the preceptor, in making a decision. The authors have also found it helpful to provide any clinical preceptor with these instructional materials to help facilitate the assignment the student will be completing under his supervision. The assignment (Table 4) encourages the student to identify the situation in which a PROM could be implemented and why and to then engage in a postencounter debrief with the preceptor. After the student implements the PROM, the student and preceptor should again debrief by reviewing the instrument outcomes together to determine how the treatment plan will be adjusted to address the specific needs of the patient. The student will need to be able to elaborate on the assignment, their reasons for choosing the PROM, and their plan for continued implementation of PROMs during Tier 4, which is the next part of the authentic assessment.
Tier 4: Reflection
Tier 4, a set of reflection questions (Table 5), is an extension of Tier 3 and is to be evaluated by the educator. The authors incorporate reflection questions for 2 reasons: (1) providing opportunities for students to reflect stimulates a metacognitive process that supports their learning of the subject matter,40 and (2) it allows the educator to evaluate the student's clinical reasoning. Both clinical reasoning and reflection assessments advocate for a learner-centered paradigm that emphasizes a shared learning experience.40,41 At the conclusion of Tier 4 the student will be able to justify the use of POEM as an integral part of evidence-based patient care. As the student becomes a more conscientious learner and knowledge seeker, the preceptor transitions from providing direct instruction to more thought-provoking and meaningful guidance.38 To promote athletic training students' transition into clinicians who practice and value patient-centered care, athletic training educators must provide students with opportunities to reflect on their clinical reasoning and accept their clinical decisions. These practices also foster future clinicians who value building relationships with their patients.41
Once students have completed their PROM integration and debriefed with their preceptor, they are asked to reflect on their clinical decisions through guided reflection questions, as outlined in Table 5. The authors have students submit a copy of the deidentified PROMs summary along with the Tier 4 reflection. The educator would then evaluate the student's ability to recall and incorporate theoretical knowledge from Tier 1, appropriately select and implement a PROM based on the patient encounter, correctly analyze the instrument outcomes, and use the instrument outcomes to drive patient care.
Clinical reflections on patient encounters are used to inform the authors how students are transferring knowledge from didactic education to clinical experience. Self-assessment of performance15 allows for the student to share what impacted his understanding of PROMs as well as his experiences with the content and pedagogical methods.14 Likewise, the educator improves her pedagogical knowledge by uncovering students' experiences and can reinforce14 content through student feedback. Over the course of 2 years, a total of 28 students enrolled in a professional athletic training program completed the activity outlined in Tier 4. Student response on the assessment amounted to identified student barriers and benefits to PROM implementation; these are outlined in Table 6. Reviewing the students' perceived barriers allows for the educator to ensure follow-up discussion with the student and preceptor if needed and to modify future lessons carried out in Tiers 1 through 3.
Despite the fact that barriers to implementation of PROMs may always exist in some form, the 28 students who completed the assessment also commented on the benefits of PROMs implementation. Students further acknowledged a mutual benefit to the patient as well as the health care provider. Just as the educator should follow up with discussions about the barriers students encountered when implementing PROMs, discussion about the positive benefits should also occur. The educator can address any areas of uncertainty and justify students' clinical reasoning. The authentic experience really allows the student and educator to improve their content knowledge of PROMs and also offers a chance to appreciate POEM in a dynamic clinical environment. This activity may also garner support and improved knowledge of PROMs use and implementation from the preceptor, depending on his previous experience and opportunities with the subject matter.
Future Considerations
As POEM has gained traction in clinical practice, there is a need for content evolution within athletic training education to properly disseminate and assess students' knowledge, skills, and abilities. The authors turned their attention to this content through commiseration about the following: a lack of meaningful assessments focused on the integration of PROMs, recognition of clinicians' perceived barriers1,5,6 to PROMs implementation, and valuation of the perceived benefits1,3–5 of incorporating PROMs in the patient-centered care model. Early author experiences of teaching the content started with basics surrounding theoretical components and then moved to requiring students to implement instruments on authentic patients during experiences. It became immediately apparent that students struggled with which instrument to choose, when to implement an instrument (ie, timing and frequency), and how the instrument would be used to support therapeutic interventions, thereby leaving the authors rethinking more effective means through which to teach the content.
While the authors continued to improve their own understanding of PROMs, one author relied on their pedagogical knowledge, while the other author relied on clinical experiences to improve the students' transfer of knowledge from the didactic to the clinical setting. This resulted in the 4-tiered approach that has been implemented and modified several times across the curriculum. Students' responses in the reflection assignment (current Tier 4) provided the authors with direction in purposefully developing each tier that is grounded in instructional evidence. Allowing adequate time to progress through and incorporate each tier lends to achievement of student learning objectives, indicative of CAATE Standards 60, 62, and 69.33
The authors have found Tiers 1 and 2 are most appropriate for didactic courses, while Tiers 3 and 4 align best with successive semester clinical courses. The course content, such as orthopaedic evaluation or therapeutic intervention, has not been found to be a determining factor in successful integration. However, after completion of all 4 tiers, PROMs have been successfully reintroduced during subsequent courses involving clinical research to improve the robustness of these assignments. Ultimately, continual refinement of this 4-tiered approach must occur to address changes in student, educator, and preceptor knowledge and clinical experiences with PROMs.
CONCLUSIONS
Learning is hierarchical; therefore, for students to learn complex content they must first obtain prerequisite knowledge and then move through subtasks of increasing complexity.42 With respect to the nature of teaching, educators must decide what to teach and how to sequence that content.43 This again highlights the fact that it is both knowledge of pedagogy and content that correlates to an educator's pedagogical content knowledge and is directly related to accomplishing student learning outcomes.7,8,11 As athletic training educators contemplate teaching and evaluating the CAATE 2020 Standards,33 it is understood that some content may be unfamiliar. Therefore, it is a collective responsibility of educators, preceptors, and administrators10 involved in athletic training programs to devote time to structured learning11,12 opportunities involving new content, collaborations between didactic and clinical entities involving patient encounters with the content,39 and utilization of reflections14 from students and peers on effective instructional methods. As POEM2 continues to be integrated into clinical practice, this 4-tiered approach (Tables 2 through 6) has been effectively used by the authors to incorporate PROMs into their respective curricula. Each tier is supported by evidence-based pedagogical techniques to include teaching styles9 and methods15,23 to guided discussion and reflection activities, which offers options to facilitate achievement of student learning outcomes.
Contributor Notes