Development and Validation of a New Competency Framework for Athletic Therapy in Canada
Competency-based education (CBE) is entrenched in educating health professionals in Canada. CBE is a framework that identifies desired performance characteristics in training competent, entry-level health professionals. To update, develop, and validate a new Canadian Athletic Therapists Association (CATA) framework for athletic therapy education. Framework development occurred in 4 phases and was developed through a multistage process that involved a scoping review (phase 1) and consensus methodology (ie, a blending of modified Ebel and modified Delphi consensus methods; phases 2–4). Phase 2: a total of 7 experts (program directors) from each Canadian accredited institution. Phase 3: a total of 14 experts (1 program director and educational expert from each accredited institution). Phase 4: a total of 7 experts (program directors) and 246 certified members of the CATA. Each phase consisted of a systematic process with 80% consensus agreement set a priori. In phase 1, a scoping review was conducted to identify common terminology that could be used to guide the framework development process and to identify competency frameworks used by other health professional organizations. Phase 2 consisted of adopting a common language that would serve to keep the expert group on the task at hand and avoid confusion. In phase 3, frameworks used by other health professional organizations were evaluated and used to determine the validity of the old CATA framework. In phase 4, the old CATA framework was updated and a new framework was developed and validated. In phase 1, the result of the scoping review yielded 368 papers, of which 5 were used to propose a common language for phase 2 and 9 highlighted competency frameworks used by other health professions for comparison in phase 3. In phase 3, the expert group voted unanimously to adopt and adapt the CanMEDS framework (ie, roles). In phase 4, the new CATA competency framework was validated, and most competencies achieved consensus. Competencies that did not achieve consensus in the first round of voting underwent face-to-face discussions via videoconferencing. After discussions, the remaining competencies were revised, and all newly worded competencies achieved consensus. The resultant framework was validated, and most competencies achieved consensus. The new athletic therapy competency framework outlines the 165 competencies resulting from this methodical process and will hopefully facilitate interdisciplinary communication and practice.Context
Objective
Design
Patients or Other Participants
Main Outcome Measure(s)
Results
Conclusions
KEY POINTS
-
Developing a new competency-based education framework for athletic therapy education in Canada is likely to facilitate interprofessional collaboration, communication, and education.
-
The adoption of a competency-based approach to health care education de-emphasizes time-based training and focuses on outcomes, abilities, and learner-centeredness.
-
The new athletic therapy framework was adapted from the CanMEDS framework and includes the following roles: athletic therapy expert, communicator, collaborator, scholar, leader, health advocate, and professional.
HISTORY OF ACCREDITATION IN CANADA
The shift from learning objectives to competency-based education (CBE) has its origins in the 1970s.1,2 McGahie et al1 first proposed this system for medical training in 1978. Since then there has been considerable momentum in implementing CBE for health professionals.3–5 CBE has now emerged as a priority topic for education planners across the health professions.6 The adoption of a competency-based approach to health care education de-emphasizes time-based training and focuses on outcomes, abilities, and learner-centeredness.6 CBE reflects the trend of defining what graduates should be able to do in practice rather than what they should know.7 CBE focuses on the end objectives of a training program regardless of time spent in education. By knowing the end product, the training program can be systematically structured to meet the end objectives.8
Many professional organizations worldwide have established consensus on competencies in the form of competency frameworks.8 Physicians are leading the shift towards competency-based medical education, so much that CBE is now entrenched in medical education across Canada, the United States, and the United Kingdom.9 CBE promotes better curricular governance, which in turn emphasizes relevant skills for medical training, leading to better health care.8 CBE has shown promising results in training entry-level health professionals capable of delivering “competent” quality patient-centered care.4,6,10 Specifically, CBE bridges the gap between theory of competencies and practical clinical work, which minimizes the disparity between expectations and realities of competence in newly graduated health care professionals, thus improving patient safety.11
Athletic therapy education in Canada has undergone significant transformation to align with trends for enhanced patient care in health professions education. In 1999, the Canadian Athletic Therapists Association (CATA), which serves as the certifying and governing body for certified athletic therapists practicing in Canada, introduced program accreditation for institutions delivering educational programs in athletic therapy.12 As part of this accreditation process, a competency framework followed the format and structure from the Board of Certification. Although this introduced competency-based training into athletic therapy education programs across Canada, this initial reform did not operate as a true CBE model because the competencies were still phrased as behavioral objectives. Much like the National Athletic Trainers' Association's previous system, an old apprenticeship-based system was retained as an external requirement in addition to completion of educational requirements to be eligible for the professional certification process. Moreover, graduates of accredited programs were required to complete 1200 practical internship hours before attempting the national certification examination. In 2007, the competency framework was updated with the primary change being a title change from behavioral objectives to competencies with few other substantive changes; however, this revision still followed the Board of Certification competency format. A CATA Education task force was formed in 2014 and made recommendations for future developments for athletic therapy education in Canada. The task force found a disconnect in delivering CBE curricula in that programs still focused on the application of knowledge rather than its acquisition.12,13 To have beneficial effects on teaching, learning, and health care, CBE must be embraced, delivered, and assessed.8 Two primary recommendations came out of the task force that are relevant to this study: (1) all accredited programs implement a formal and comprehensive CBE model by the year 2020; and (2) the removal of the 1200 hours as an external requirement for certification eligibility.13 Therefore, the purpose of this study was to develop and validate a competency framework for athletic therapy education in Canada, which includes a comprehensive CBE that would allow elimination of the requirement for 1200 clinical hours. The outcome of this process aligned the athletic therapy competency framework with national and international CBE standards.
METHODS
Study Design
We struck the steering committee that also included a group educational experts to guide the competency framework development process. This steering committee was involved in all 4 phases of this study and exclusively in the first 3 phases:
-
Phase 1: Scoping review (2 months)
-
Phase 2: Proposing a common language (2 months)
-
Phase 3: Evaluating existing competency frameworks (2 months)
-
Phase 4: Athletic therapy competencies validation (18 months)
The phases were iterative and built on each other successively. The final athletic therapy competency framework was generated in the final phase (phase 4). Phases 2 to 4 used a consensus process that was a blend of 2 consensus methods: a modified Ebel procedure and a modified Delphi approach.14 Each phase consisted of a systematic process with 80% consensus agreement set a priori. The consensus groups were different for various phases and are described in greater detail in the following paragraphs. Within each phase, draft documents were circulated, followed by an oral presentation of material (through videoconference technology), a blinded vote, and subsequent virtual face-to-face discussion to debate items that did not achieve 80% consensus agreement. Comments and feedback during the face-to-face discussion was facilitated by the primary author (M.L.). Items were discussed until at least 80% consensus was reached. All phases of this research were approved by the Mount Royal University Human Research Ethics Board.
Phase 1: Scoping Review
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was adapted for the scoping review.15 In contrast to a systematic review, which tends to include a narrow range of study designs, the scoping review was selected to address this broad topic and allow for various study designs. The goal of the scoping review was 2-fold: to identify common terminology that could be used to guide the CATA competency framework development process (phase 2) and to identify competency frameworks and competencies used by other health professional organizations (phase 3). MEDLINE and CINAHL were searched to identify relevant studies. Searches covered the time period from database inception to December 2018. The search strategy combined text terms and medical subject headings (MeSH) using the terms “competency-based education AND (framework or language)” in any field. Abstracts were analyzed to identify studies that met inclusion criteria for full-text review. In cases where it was not possible to determine whether inclusion criteria were met, full-text manuscripts were also retrieved. The reference lists of manuscripts that met inclusion criteria were manually searched to identify additional studies. Gray literature and Google were also searched using the same terms.
Manuscripts were limited to studies that described CBE frameworks in the context of health professions with similar scopes of practice to athletic therapists such as athletic trainers, physiotherapists/physical therapists, chiropractors, and occupational therapists. Medical education frameworks were also included because of the leading role of medical education in developing and implementing CBE.9 The search was restricted to English-language literature. Additional inclusion and exclusion criteria specific to phases 2 and 3 are subsequently defined.
Phase 2: Proposing a Common Language
The steering committee identified the importance of adopting a common language in the competency-framework development process. This common language served to keep the expert group on the task at hand and avoid confusing competencies with behavioral objectives and task statements that were developed in the 2007 revision process. Manuscripts from the scoping review were used to provide a starting point for development of a common language. Manuscripts were included if they presented universal terminology, a glossary, or accepted definitions currently being used in the CBE domain. Specifically, this included terminology that had reached consensus and had been adopted by other health professions pursuing CBE to ensure that the “community proceeded on the same page.”16 Two members of the steering committee (M.L., J.O.) screened all manuscripts by a full-text review. Data extraction included author(s), year, title, study design, and relevant terminology. Subsequently, this information was compiled into a working document and presented to an educational-expert group. The educational-expert group consisted of program directors from 7 accredited programs in Canada. All educational experts (hereafter, experts) had been teaching in accredited programs for at least 10 years and as long as 39 years. Descriptive data for these experts are listed in Table 1. Experts were asked to rate each definition on a 2-point scale (agree or disagree). Consensus was reached when 80% of experts agreed.

Phase 3: Evaluating Existing Competency Frameworks
In this phase of the research, expert consensus was sought to adopt a new competency framework that was identified in phase 1 of this project (the scoping review). Frameworks were excluded if they were developed for subspecialties within a profession (eg, internal medicine), if they were specific to clinical diagnoses (eg, diabetes, cancer), or if competencies were organized by domain (eg, patient care, medical knowledge, systems-based practice). Manuscripts discussing competency-based assessment/evaluation, program design, or framework implementation were also excluded. Interprofessional frameworks were also excluded. Two members of the steering committee (M.L. and J.O.) screened all remaining manuscripts by a full-text review that met the initial criteria identified in phase 1 while also meeting the exclusion criteria. Manuscripts were summarized and the following data was extracted: author(s), year, title, country of study, corresponding health profession, and competency framework. A working document was compiled from the data and presented to the expert group. In each expert was asked to recommend an educational expert from their home institution to participate in an additional round of voting. These other experts were required to hold an academic appointment and have at least 10 years of postsecondary teaching experience. Descriptive data for these experts are also listed in Table 1. After the presentation, the expert group discussed the strengths and weaknesses of each framework and whether any of the frameworks would meet the needs of CATA. Using Qualtrics survey software, a blind vote was conducted, and 14 experts (2 from each of the 7 accredited programs) voted using a 100-mm visual analog scale that was anchored with not important and extremely important. All 14 experts were instructed to vote on the importance of agreeing to adopt: (1) the selected framework; (2) role competency categories within the framework; and (3) definitions for each role.
Phase 4: Athletic Therapy Competencies Validation
The steering committee began this phase with the creation and refinement of the new competencies. Content and language from the existing competencies and task statements were updated for language, consistency, and appropriate verb allocation.17 A Bloom's taxonomy verb guideline was used to assist with progressive cognitive skill development ranging from simple acquisition of knowledge to complex thinking skills.18 Other taxonomies exist that focus on the relationships among learning outcomes, learning activities, and assessments.19,20 An important step in this process was to update the previous competencies using measurable verbs that best fit the previously identified domain while integrating them into the new competency roles. Another important variable in this process was to compare and ensure consistency around language use between the old CATA framework and the frameworks identified in phase 3 of this project. Once the steering committee was satisfied with the new draft, a 2-stage consultative approach was implemented using the CATA membership and the expert group.
In the first stage of consultation, the CATA membership was sent an initial draft of competencies broken down by competency role. This stage functioned as an initial screen of the competencies to ensure there was accurate representation of the competencies across the full breadth of employment settings of athletic therapists in Canada. The CATA membership was asked to use a 100-mm visual analog scale that was anchored with extremely important and not important. The other measure that was collected but was not used for content validation purposes was the frequency with which members used each competency. The steering committee used the data to screen the competencies and identify major gaps that could be addressed before being sent to the expert group for voting. Each role was sent out to the CATA membership for voting and spaced approximately 2 to 3 weeks apart.
In the second stage, results from the CATA membership for each role survey were reviewed by the steering committee. These results led to a few minor revisions before the competencies were sent to the expert group. The expert group was asked to rate 3 things: (1) the importance scale; (2) the frequency scale; and (3) a binary measure to keep or remove a specific competency. Competencies that achieved both 80% consensus to “keep” and a mean score of 80 (on a 0–100 scale) across experts for importance were automatically retained as consensus competencies. Competencies that did not achieve 80% consensus or where at least 2 expert group members voted to remove the competency were reevaluated.
RESULTS
Phase 1: Scoping Review
The search yielded 368 manuscripts, of which 14 manuscripts were selected for full-text review (Figure 1). To establish a common language, 4 manuscripts met inclusion criteria from database searching based on titles and abstracts, and one additional manuscript was identified from manually searching the reference list. To identify competency frameworks used by other health professional organizations, 6 manuscripts met inclusion criteria from database searching and were selected for full-text review. After searching gray literature and Google, an additional 3 frameworks were included. A detailed summary of manuscripts included in the review can be found in Appendix A.



Citation: Athletic Training Education Journal 16, 1; 10.4085/1947-380X-20-080



Citation: Athletic Training Education Journal 16, 1; 10.4085/1947-380X-20-080
Phase 2: Proposing a Common Language
Five manuscripts were included and resulted in the identification of 5 seminal terms.16,21–24 These terms were regarded as important concepts towards a common language and shared understanding of CBE: (1) competency-based education; (2) competency; (3) competence; (4) entrustable professional activities (EPAs); and (5) milestones. In addition, the expert group proposed the addition of 1 more term that was adopted from the CATA program accreditation manual: national standards of practice.25 All terms and respective definitions unanimously reached consensus (ie, 100% agreement). All terms and definitions are presented in Table 2.

Phase 3: Evaluating Existing Competency Frameworks
The scoping review identified 6 manuscripts from the search strategy26–31 that highlighted 3 frameworks that met inclusion criteria: CanMEDS, Scottish Doctor, and the competency profile for the entry-level physiotherapist in Canada (adapted from CanMEDS). An additional 3 frameworks were identified from a Google search: athletic training education competencies (National Athletic Trainers' Association),32 core competencies of the chiropractic specialist in physical and occupational rehabilitation (adapted from CanMEDS),33 and sports physiotherapy competencies and standards.34 All 6 frameworks were presented to the expert group. After discussing strengths and weaknesses of each framework, it was proposed that the CanMEDs framework would suit the needs of the CATA best.
The initial voting results were unanimous for the adoption of the CanMEDS framework and all 7 role competency categories within the CanMEDS framework: medical expert, communicator, collaborator, health advocate, leader, scholar, and professional. The expert group also agreed to change medical expert to athletic therapy expert. Five of 7 role definitions achieved consensus. Athletic therapy expert and collaborator failed to achieve consensus, thus a face-to-face discussion via videoconferencing technology was required. The discussion was facilitated by the primary author (M.L.), and experts were encouraged to state concerns with the current definition. Conceptual consensus was reached for these role definitions in the videoconference. Using the conceptual consensus, the steering group formulated a new definition, which was circulated to the expert group for a second vote. Subsequently, both revised definitions for athletic therapy expert and collaborator were approved with over 80% consensus. All 7 roles and definitions are presented in Table 3. Upon sharing the results of the second vote, the steering committee invited members of the expert group to create a visual representation that encapsulated the new competency framework using the CanMEDs framework for inspiration. Only 6 submitted a hand-drawn representation, all with varying conceptual designs. The steering committee met and decided on the final design, which resulted in an artist's rendition of the new visual representation (Figure 1). The resulting image depicts a maple leaf in which 2 roles are central to how athletic therapists have traditionally practiced: athletic therapy expert and professional. These central roles are highlighted in grey (central roles), whereas the other roles are in white. However, each lobe of the leaf portrays a different complementary role that nourishes the technical expertise of the certified athletic therapist as an independent practitioner. The web of veins that connects all the roles of a certified athletic therapist illustrates a complex network, thus contributing to holistic and integrated competence.

Phase 4: Athletic Therapy Competencies Validation
During the first stage of consultation with the CATA membership (n = 2700), the response rate varied on the basis of the role being surveyed (Table 4). The majority of competencies achieved the 80% threshold, and there were only minor editorial changes to the wording of most competencies at this stage. It should be noted that the primary reason for this stage was to identify missing items or competencies and to confirm breadth of competencies. There was only 1 suggestion across all competency roles for an additional content area: concussion. As a result, a new competency was added under the athletic therapy expert role related to concussion recognition and intervention. Data from the membership were consulted as an additional mechanism of feedback and wording for those items that did not achieve consensus with the expert group.

Consensus among the experts was achieved for the majority of competencies in the first round of voting. Table 5 outlines the results from the first round of blinded voting broken down by competency role. Competencies that did not achieve consensus or that had at least 2 members suggest the competency be deleted were discussed during the face-to-face videoconference. Discussion ensued, resulting in rewording of the competencies, which was communicated through a shared-screen function. All competencies that did not achieve consensus in the first round of voting achieved unanimous consensus in the second round. The final competencies can be found in Appendix B.

DISCUSSION
The goal and primary outcome of this study was to develop and validate a new competency framework for athletic therapy education in Canada. The athletic therapy competency framework was developed through a multistage process that involved a blending of the modified Ebel and modified Delphi consensus methodologies.14 These methodologies were used to evaluate existing frameworks, resulting in the adoption and subsequent adaptation of the CanMEDS role-based framework. The CanMEDS framework has been adopted by many other medical programs internationally.36 More important, this framework was adopted by several allied health care professions in Canada including physiotherapy, chiropractic, and occupational therapy.28 The greatest overlap in scope of practice with athletic therapy in Canada is primarily with medicine, physiotherapy, and chiropractic. The original Canadian athletic therapy competencies acted as the largest foundation for the development of new competencies, but language and concepts were pulled from CanMEDS, Canadian physiotherapy, and Canadian chiropractic education frameworks. Therefore, the adoption of the CanMEDS model for athletic therapy education in Canada will facilitate interprofessional collaboration, communication, education, and practice, in accordance with other health professional organizations in Canada.28,35
Klamen et al37 identified that the most important tenet of CBE is the importance of sufficient description, in advance, of what needs to be learned. Adopting a common framework is an important outcome of this study. Verma et al28,35 shared the importance of creating competency frameworks that not only meets the needs of individual health care professions but also helps to facilitate interprofessional collaboration, communication, and education. The framework and language used in the new athletic therapy competencies should help situate the profession in Canadian health care and, ideally, internationally as well. Accredited athletic therapy programs in Canada set a target to deliver CBE by the year 2020.13 The new athletic therapy competency framework, the associated roles, the definitions and associated competencies were presented to the CATA in July, 2019. The report that outlined the process and results of this work was ratified and accepted as the new competency framework for the CATA by the board of directors in August 2019. The competencies were officially adopted into the standards used to accredit programs in Canada commencing in 2020, thus meeting the implementation target originally set out in 2016.
One challenge that the new Canadian athletic therapy framework will create is related to the standards and language across international boundaries. In 2018, Izumi and Tsuruike38 compared athletic therapy/training globally. The primary terminology used in this analysis was tasks, which would be akin to EPAs in the new framework presented in this study. As a result of these differences, using mutual recognition agreements such as those previously outlined for international athletic training/therapy will need to be carefully compared and analyzed.39 The process to determine equivalency will require some translation to ensure the underpinning spirit of competencies and EPAs are comparable internationally if credential recognition is considered.40
A similar pathway to these kinds of challenges around language has been carved in medical education. Englander et al16 attempted to provide greater clarity by establishing a shared language among medical educators and researchers. In the current study, we have chosen to adopt a similar language to that presented by Englander et al16 to facilitate a broader yet consistent understanding of the concepts and theories that comprise CBE.
Limitations
The project took place over a two-and-a-half-year period, and as a result, there was some turnover in experts at various phases according to availability to commit to the voting and discussion required. Seven experts who served in the expert group changed between the first 3 phases and the last phase due to medical or professional leaves of absence. However, the experts who replaced the original program director also met the criteria of expertise with more than 10 years of athletic therapy educational experience.
Another limitation is attributed to the loss of subject anonymity with the blending of the modified Ebel and Delphi procedures. Subject anonymity has the potential to reduce the influence of dominant or authoritative individuals or vice versa. Videoconferencing, however, facilitates a face-to-face meeting and allows the expert group to seek clarification for difficult subject areas that would otherwise not reach consensus. In addition, the primary author (M.L.) attempted to facilitate the face-to-face discussions in such a manner that all parties needed to speak for or against wording of competencies.
One other limitation is respondent fatigue, which led to a decrease in response rates from the CATA membership as subsequent surveys were distributed. Additional context for the data presented was that the response rate was based on the total membership (ie, 2700). However, approximately 700 of those members were students and likely did not participate in the research, although they were all sent an invitation to do so. Although this total reduction in responses could lead to biased results, surveying the CATA membership was a part of a 2-stage consultative approach whereby the expert group contributed to content validation in the second phase, thus ensuring breadth of competencies was attained. Another important limitation is that the framework developed has expert consensus and is a solid representation of current literature, but its development does not ensure that it will work best for education. Validation may only be revealed once students are instructed with this framework in mind and are emerging as competent health care providers.
CONCLUSION
The new athletic therapy competency framework outlines the final competencies resulting from this methodical process. It should be noted that these competencies are contemporary and appropriate as of 2020. However, this is the first edition of the athletic therapy competency framework and future revision and renewal should be undertaken in approximately 7 years (by 2027) to ensure these competencies continue to accurately represent the evolving athletic therapy scope of practice in Canada.

PRISMA flow diagram of the identified studies for the scoping review.

A visual representation of the new athletic therapy competency framework.
Contributor Notes