Purposeful Course Planning: Considering Student Self-Efficacies When Selecting an Online, Hybrid, or Face-to-Face Course Delivery Modality
Self-efficacy (SE) can affect athletic training students' progression during their professional education and transition to autonomous clinical practice. It is unclear how course delivery may affect athletic training students' SE in various injury evaluation courses. Determine the relationship of course delivery modality and athletic training students' injury evaluation SE. Cross-sectional investigation. Web-based survey. Ninety-five noncertified National Athletic Trainers' Association student members (38/95 undergraduate athletic training students; 57/95 graduate athletic training students). A 2-part survey including participant characteristic questions and piloted adapted General Self Efficacy (GSE) scales were distributed using the National Athletic Trainers' Association's Research Survey Service. The adapted GSE scales asked participants to rate their agreeance from 1 (not at all true) to 4 (exactly true) for 10 statements focused on injury evaluations in 6 areas. Descriptive statistics were calculated, and nonparametric tests were used to determine the differences in GSE scores based on course delivery modality (online model, hybrid model, and traditional face-to-face model). Measures of central tendencies were also calculated. Only completed surveys (66.4%; 95/143) were included in the analysis (access rate = 14.3%). A significant difference existed between course delivery modality and upper extremity GSE scores (P = .001). No significant differences were found between GSE scores for courses with delivery modalities affected or not affected by the COVID-19 pandemic as well as between athletic training students who had or did not have previous online or hybrid course experiences. As health care education continues to shift toward technology-rich environments, educators can consider offering courses through various delivery modalities to promote didactic and clinical education. However, consideration of the content area and difficulty may be warranted while purposefully planning courses to best address learning objectives and students' SE.Context
Objective
Design
Setting
Patients or Other Participants
Data Collection and Analysis
Results
Conclusion(s)
KEY POINTS
-
It is important for athletic training educators to consider how course content (eg, anatomical region, difficulty of content) and course content delivery (eg, online, hybrid, face-to-face modalities) may affect student self-efficacy and clinical performance.
-
Previous online or hybrid course experience was not associated with differences in athletic training students' orthopedic injury evaluation self-efficacies.
-
Course delivery modality changes during the COVID-19 pandemic was not associated with a change in athletic training students' self-efficacies for orthopedic injury evaluations.
INTRODUCTION
Technology-rich education even before the COVID-19 pandemic required distance learning has become increasingly prominent in health care education, such as in athletic training programs,1,2 nursing programs,3,4 and medical schools.5–7 To provide an educational experience for athletic training students that aligns with the learner-centered, technology-rich trends of today's higher education, course delivery methods used in athletic training education may also need to adapt.8,9 Courses that integrate the use of technology, such as fully online instructional courses or hybrid courses which blend online instruction with in-person practice, may allow more time for hands-on practice and clinical experience, which might better prepare students for a more successful transition to autonomous clinical practice.8
Moffit and Lindbeck2 investigated the current uses of technology in athletic training education and found no true pedagogical differences. Although faculty were integrating technology into courses, such as the use of YouTube videos and PowerPoint lectures, faculty still relied on traditional lecture-style classrooms that did not promote student-centered, case-based learning, or increase hands-on practice time.2 Students in other health care programs reported that online learning can be beneficial but hinged on whether the educator was ready and knowledgeable to facilitate an online course.7,9 Specific to athletic training, Winkelmann and Eberman9 found that students who did not have previous online learning experiences had lower self-efficacy (SE) and acceptance of technology in the classroom before being exposed to an online learning environment. Courses that are designed to provide problem-based, learner-centered experiences can encourage students to critically think and become self-directed learners.1,10,11 By encouraging students to create knowledge on their own, they may be able to better translate the learned skills into clinical practice.11
The requirement to abruptly shift course delivery from an in-person to a virtual format during the COVID-19 pandemic has affected health care education programs across the world.5–7 For example, medical students who were removed from their clinical rotations and transitioned to online classes reported a loss in hands-on practice of skills and mentorship experiences.7 These students also reported that online learning was beneficial for some but not all topics, specifically, those that required those hands-on components.7 Increased demands for more health care students to enter the workforce required many programs to quickly adapt to continue didactic and clinical education6 and troubleshoot new challenges, such as insufficient connectivity at home.12 These adjustments, such as increased virtual or online learning and assessments, may be part of the new normal in health care education, but specific investigations in athletic training programs are warranted.6,7
Course delivery modality has been identified as one variable that affects students' perceived SE.1,13,14 Self-efficacy, as defined by Bandura,15 is the confidence of or belief in a person's capabilities in a specific environment and is based on an individual's interpretation of 4 constructs: mastery experiences, vicarious experiences, verbal or social persuasion, and emotional or physiological states. Mastery or enactive experiences are the successful executions of tasks that show an individual his or her own capabilities and limitations; the construct of vicarious experiences is based on the idea that SE improves when observations occur of success in role models or others that an individual can relate to; verbal or social persuasion is the concept that verbal encouragement from significant people in an individual's life may affect that individual's SE; and emotional or physiological states are the positive and negative emotions an individual experiences that may increase and decrease self-confidence and SE.15,16
Understanding Bandura's Self-Efficacy Theory15 and how the 4 constructs affect SE is vital to fostering environments for students to excel.17 More time spent practicing skills can increase students' confidence17 and SE, which is also a predictor of students' clinical performance.18,19 For example, decreasing in-person class time spent on content delivery and increasing students' hands-on practice time with instructor feedback may increase students' confidence and SE in those skills. Students have reported a lack of practice, lack of confidence, and lack of positive mentor influences as perceived barriers in their ability to accurately perform an injury evaluation.20 Therefore, students' low SE may affect their ability to successfully transition to practice,21 where they must perform clinical skills, make clinical decisions, and subsequently practice as autonomous clinicians.22,23 However, limited research exists on athletic training students' SE regarding injury evaluation skills with relation to current course delivery modalities. Therefore, the purpose of this study was to examine the relationship between course delivery modality and athletic training students' orthopedic injury evaluation SE.
METHODS
Study Design
A descriptive, cross-sectional study design was used to assess the SE ratings for orthopedic injury evaluations of a randomized sample of athletic training students. The checklist from the Strengthening the Reporting of Observational Studies in Epidemiology statement for cross sectional studies24 was used while designing the study and adapting the instrument. The A.T. Still University Institutional Review Board deemed this study exempt.
Operational Definitions
For this study, the following definitions for the subcategories of the course delivery modality variable were used.
Hybrid (Hybrid Model)
A hybrid model uses a combination of face-to-face and online activities which may include students' use of technology to asynchronously obtain knowledge or lectures and synchronous face-to-face class sessions for discussions, interactions, hands-on activities, or all the above.1,12,25
Online (Online Model)
Online education models heavily rely on technology resources to produce a virtual or semivirtual classroom.26
Traditional Face-to-Face Model (F2F)
Traditional face-to-face models use traditional teacher-centered lectures as the primary educational technique.26
Instrumentation and Data Collection
Schwarzer and Jerusalem's27 General Self Efficacy (GSE) scale asks the participant to rate their agreeance with 10 SE statements as 1 (not at all true), 2 (hardly true), 3 (moderately true), or 4 (exactly true). The GSE scale has been validated in over 23 languages, resulting in Cronbach α ranging from 0.75 to 0.91.28 With over 1000 studies in which authors have included the use of this scale, Scholz et al28 assured us that the GSE is a scale that is unidimensional, internally consistent, and stable over time through psychometric testing. Criterion-related validity was also documented with its use in numerous correlation studies.27–29
To examine athletic training students' SE of injury evaluations, we used a 2-part survey. Part 1 of the survey included participant characteristic questions. Part 2 included the adapted GSE scale for orthopedic injury evaluations. This broad skill was split into 6 categories: (1) general orthopedic injury evaluation, (2) upper extremity musculoskeletal injury evaluation (UE), (3) lower extremity musculoskeletal injury evaluation (LE), (4) head or neck injury evaluation (HN), (5) thorax injury evaluation (TX), and (6) spine injury evaluation (SP). We made minor adaptations to the GSE items by adding in or replacing general terms (ie, difficult problems) with a reference to injury evaluations based on these categories. For example, if the GSE item stated, “It is easy for me to stick to my aims and accomplish by goals,” the adapted GSE item was revised to, “It is easy for me to stick to my aims and accomplish my goals during an upper extremity musculoskeletal injury evaluation.” An example of terminology replacement in a GSE item stem included the change from, “I can solve most problems if I invest the necessary effort,” to, “I can work through most musculoskeletal evaluations if I invest the necessary effort.”
Content Expert Review and Pilot Testing
These minor adaptations underwent expert review (ie, 4 experienced athletic training faculty and researchers) for clarity and comprehensiveness. Only minor modifications to the wording of the participant or program characteristic questions were suggested and made after the content expert review. The researchers then distributed the adapted survey instrument to a pilot sample of athletic trainers and experienced preceptors (n = 13) who were asked to reflect on their SE as students or current observations of students learning to evaluate orthopedic injuries. We piloted the survey with this population instead of students to avoid overlap in future sampling but also to use their familiarity with orthopedic injury evaluations and applicability to clinical practice, not just coursework. The main constructs of the original GSE scale did not differ from the constructs in the adapted GSE scale, and only participant characteristic items were reordered or modified after expert review and piloting; therefore, no further validation or psychometric testing were completed on the adapted GSE scale before distribution.
Data Collection
After receiving approval from A.T. Still University Institutional Review Board, student members received an email requesting their voluntary completion of the anonymous survey and explaining they could choose to discontinue their participation at any point. The email also contained a summary of the study, researcher (M.F.) contact information, and the URL to the Web-based survey. The e-mail was sent in December 2020, with reminders every 2 weeks to complete the survey through January 2021. Participants provided consent through the act of completing the survey and received no direct benefits from participating in this study.
Participants and Recruitment
The National Athletic Trainers' Association (NATA) Research Survey Service was used to recruit participants, distribute the survey, and collect data from noncertified student members. As of December 2020, of the 4598 noncertified undergraduate and graduate student members of the NATA, 4354 opted in to receive research surveys. An a priori power analysis was conducted using G*power30 for a linear multiple regression model using a 2-tailed test, a small effect size (d = 0.10), and an α of .05. The result indicated that a total sample of 81 participants was required to achieve a power of 0.80.
The NATA randomly distributed the 2-part, Web-based survey by e-mailing 1000 noncertified student members. Of the 1000 participants invited to complete the survey, 143 agreed to participate by accessing the online survey (access rate = 14.3%). Surveys with incomplete responses to all GSE scales were omitted (n = 48/143, 33.6%) since it was a major component of the research aim. Therefore, 95 completed surveys (completion rate = 66.4%) were included for analysis. Participants had an age of 23.04 ± 2.40 years, were mostly female (70/95, 73.7%), and enrolled in a professional graduate-level program (57/95, 60.0%). An overview of the survey adaptation process and participant recruitment methodology are presented in the Figure. Additional participant and program characteristics are presented in Tables 1 and 2.
Citation: Athletic Training Education Journal 17, 3; 10.4085/1947-380X-21-038
Data Analysis
After conducting statistical analyses, the course delivery modality groups were collapsed into 3 groups: (1) hybrid, which included the hybrid model and traditional face-to-face model that changed to a hybrid model due to COVID-19 pandemic (F2FHybrid); (2) online, which included the online model, traditional face-to-face model that changed to a fully online model due to COVID-19 pandemic (F2Fonline), and hybrid model that changed to an online model due to COVID-19 pandemic (HybridOnline); and (3) traditional face-to-face (F2F). SPSS Version 27 (IBM) was used to calculate descriptive statistics, inferential statistics, and to analyze the dependent variable data through histograms to determine any outliers. Kruskal-Wallis H-tests were used to determine (1) if differences existed in GSE scores between course delivery modality groups for each injury evaluation course and (2) if differences existed in UE, LE, HN, TX, and SP GSE scores based on the number of previous online and hybrid courses. When Kruskal-Wallis H tests revealed significantly different GSE scores, pairwise comparisons with a Bonferroni correction were then used to determine differences between course delivery modality groups. Independent-sample t tests were conducted to determine if differences existed in UE, LE, HN, TX, and SP GSE scores between students whose course delivery modality changed and students whose course delivery modality did not change in the spring, summer, and fall 2020 semesters. No outliers were identified with these data.
RESULTS
Course Delivery Modality and GSE Scores
Mean UE, LE, HN, TX, and SP GSE scores are presented in Table 3, and the results of the Kruskal-Wallis H tests used to determine differences in GSE scores based on course delivery modality are presented in Table 4. Distributions of GSE scores were similar for all groups, as assessed by visual inspection of a boxplot. Upper extremity GSE scores were statistically significant based on different course delivery modality, χ2(2) = 13.63, P < .001. Pairwise comparisons with a Bonferroni correction revealed statistically significant differences in UE GSE scores between the online (mdn = 27.00) and F2F (mdn = 32.00; P < .001) course delivery modality groups but not with the hybrid course delivery modality group (mdn = 30.00). The η2 estimate of effect size for UE GSE scores was 0.168, which is categorized as a large effect.31 No significant differences existed between GSE scores for any other injury evaluation courses based on course delivery modality.
Previous Online or Hybrid Course Experience and GSE Scores
Kruskal-Wallis H tests were used to determine the differences between UE, LE, HN, TX, and SP GSE scores for the 6 groups of online and hybrid course experiences (0–5+ previous online or hybrid courses). Only participants who completed courses through hybrid or online course delivery modalities before spring 2020 were included in this analysis. Distributions of GSE scores were not similar for all groups of previous online course experiences, as assessed by visual inspection of a boxplot. No significant differences existed between any GSE scores based on number of previous online or hybrid courses (Table 5).
Effect of COVID-19 Pandemic
For this analysis, course delivery modalities were reorganized into 2 groups based on whether a change or no change in modality occurred during the spring, summer, or fall 2019 semesters: (1) course delivery modality did not change, which included the hybrid, online, and F2F course delivery modalities groups and (2) course delivery modality changed which included the F2FHybrid, F2FOnline, and HybridOnline course delivery modality groups. Only students who completed their injury evaluations courses in spring, summer, or fall 2020 were included in this analysis. The number of participants who completed each course in spring, summer, or fall of 2020 and whether their courses changed delivery modality in 2020 due to the COVID-19 pandemic are presented in Table 6. The results of the independent-sample t tests conducted to determine if differences existed between UE, LE, HN, TX, and SP GSE scores between students whose course delivery modality changed or did not change are presented in Table 7. No significant outliers were observed, as assessed by inspection of a boxplot. All GSE scores were normally distributed (except for LE GSE scores, P < .001), as assessed by the Shapiro-Wilks test (P > .05), and homogeneity of variances occurred for all groups, as assessed by the Levene test for equality of variances (P > .05). No statistically significant differences were found between students who reported their course modality changed or did not change for any of the courses.
DISCUSSION
The aim of this study was to determine if a relationship existed between orthopedic injury evaluation course delivery modality and athletic training students' SE for orthopedic injury evaluations. We hypothesized course delivery modality, previous online or hybrid course experience, and the abrupt change in course delivery modality due to the COVID-19 pandemic would affect athletic training students' GSE scores. However, the results of this study revealed (1) only UE GSE scores were associated with a change in course delivery modality, and (2) previous online or hybrid course experience and abrupt changes to course delivery modality due to the COVID-19 pandemic did not affect GSE scores.
Course Delivery Modality and GSE Scores
In this study, only UE GSE scores were significantly different between online and F2F instruction. It is unclear why only UE GSE scores were affected by the course delivery modality; however, it is possible that delivery of factual and procedural knowledge in health care education may affect students' GSE in some areas more than others. Factual knowledge is the theoretical background information that is abundant and may be difficult for students to obtain. Procedural knowledge is the application of medicine to real life circumstances and must be practiced,32 which can contribute to students' mastery experiences. For example, students may have found UE injury evaluations required the use of more factual knowledge and were procedurally more difficult to complete, and therefore, the way the content was delivered to students (online, hybrid, or F2F) may have affected their SE. As such, UE GSE scores may have been higher for a F2F course since students could receive immediate answers to their questions and in-person feedback, which would contribute to improving students' SE through mastery experience. Students in online courses may not have fully obtained the factual and procedural knowledge or may have experienced a delay in hands-on practice time or feedback, which may have affected their UE GSE. However, it is not surprising that no differences existed in UE GSE between a hybrid and F2F delivery since a hybrid approach could have resulted in more time dedicated toward hands-on practice of skills (mastery experiences) with feedback (verbal persuasion), which could improve students' SE.15 Therefore, tracking online student engagement, periodic testing of students' factual and procedural knowledge, and providing timely and routine feedback for hands-on skills may help improve students' SE when using an online format for UE coursework.
Previous Online or Hybrid Experience and GSE Scores
Winkelmann and Eberman9 found that athletic training students had lower SE before being exposed to an online environment; however, we found that previous online or hybrid course experience had no effect on GSE scores. Although, in the current investigation, we did not inquire about the content area of previous online or hybrid courses taken, it is possible that our participants were more familiar with online educational platforms or that the courses they took in the past did not include clinical skills. For example, students' previous online course experiences could have been online English or psychology courses, which would not require clinical problem-solving or hands-on skills practice. Therefore, students' previous online course experience, without the mastery experiences related to successfully executing a specific task,15 may not have affected their perceived SE for orthopedic injury evaluations. It is possible that mastery of the course content may be different for other courses compared with orthopedic injury evaluation courses.
Although previous online or hybrid course experience was not associated with a difference in SE in the current study, advantages and disadvantages exist for educators and students depending on the course delivery modality used for athletic training courses. Curriculum development, instructional planning, and course setup for online and hybrid courses can be time consuming for educators,33 particularly for those who are new to teaching or new to using an online educational platform. However, educators' time burden may decrease with each subsequent offering, and they can then focus on creating additional resources, such as instructional videos with embedded quiz questions, to increase student engagement with the material and to make clinical connections. However, as more online and hybrid courses may be offered, especially with the ongoing pandemic, the likelihood of students and educators having previous online, or hybrid course experiences will likely increase.
Effect of COVID-19 Pandemic
Although the COVID-19 pandemic forced an abrupt change in course delivery modality for many, no significant differences existed in GSE scores of students who did or did not take courses that were transitioned online due to the COVID-19 pandemic. It is possible that students could still engage in some clinical experiences (eg, sites that remained opened, high-fidelity simulation experiences, virtual clinic scenarios) to improve or maintain SE, despite a potential change in the delivery of the instructional content. It is also possible athletic training students and educators quickly adapted by creating virtual clinics, sharing online resources through NATA's AT EducATionalist Community, and using established online resources already in place at many universities. Many of these creative technological solutions can still be used postpandemic and allow for cross-program collaborations to help program administrators address areas of need. For example, video conferencing has become commonplace and can connect students across programs with experts across the nation and abroad. However, as delays in teaching and applying clinical skills have been reported by some medical programs6 and likely athletic training programs too, ongoing investigation of students' SE and their transition to practice is warranted.
LIMITATIONS
Despite the contributions of this study to the current literature, findings should be interpreted with caution. Generalizability is limited, given that, in this study, we included a small sample of athletic training students. Although sample size was small, it satisfied the power prediction and was a representative sample of athletic training students across the nation. It is also possible that students who completed the survey may have been those interested in discussing their own SE, and students with higher SE may have been more likely to complete the survey. The results from this study may still contribute to generating timely dialog among educators regarding course delivery modality decisions and students' SE as technology use increases within academia. The reported GSE scores may also only represent students' SE in relation to instructional courses and may not fully capture the role of their clinical experiences in their SE. Whereas in this study we used a small effect size to determine significance, more meaningful differences in clinical practice may require using a larger effect size and sample to evaluate students' SE throughout their time in the program or as they prepare to transition to practice. However, in this study, we did evaluate a snapshot of students' SE, which may change throughout their time in the program and based on the opportunities to practice these skills in the field. A larger effect size would be better suited for a study in which authors evaluate student SE over a longer period of time or throughout the students' time in their program.
FUTURE RESEARCH
Future researchers should examine how clinical experiences and ongoing technology use for instruction during COVID-19, coupled with different course delivery modalities, may affect (or not affect) student SE. An in-depth understanding of how instructors use technology (eg, supplemental instruction, flipped classroom, tool for additional feedback) or the role of virtual clinics and high-fidelity simulations is needed to better inform educators' decisions regarding how and when to modify course delivery and best practices for technology use in athletic training education. Additionally, investigation of most effective pedagogical strategies to improve students' factual and procedural knowledge, hands-on clinical skills, and clinical decision making is warranted since the use of technology in higher education and professional development is likely to continue.
CONCLUSIONS
Even though the COVID-19 pandemic may have forced many courses online or into hybrid formats, this shift toward technology-rich education in higher education was already trending in that direction and may continue in a post-COVID-19 pandemic world. Therefore, consideration of course modality, technology use, and its influence of students' SE is increasingly important. Overall, in this study, we found limited differences between course modality and students' SE while evaluating orthopedic injuries. Technology can be used to deliver content but can also be strategically used to yield more opportunities for hands-on or mastery experiences and immediate feedback to improve students' SE. While planning future coursework, educators should reflect on the content being taught, course sequencing in the program, requirements for clinical experience hours, and students' technological skills when determining which course delivery modality will best meet students' needs. Ongoing evaluation of course delivery modalities and students' SE may also help to make future programmatic changes to adapt to the new normal of online or hybrid learning in health care education while minimizing detriment to students' SE.
Methodology and survey flow chart.
Contributor Notes