Actual and Perceived Questions Asked by Preceptors with and Without the Use of Bug-in-Ear Technology

Design

I used a convergent parallel mixed-methods research design to conduct this study.¹⁶ The study was grounded in a qualitative approach, including field observations and interviews, to gain multiple perspectives of the questions asked during clinical education experiences.¹⁷ Field observations and audio recording of preceptor-student conversations produced transcriptions of the questions asked by preceptors, which were coded qualitatively and then analyzed quantitatively. After interviews and questions were analyzed independently, results were combined for discussion and interpretation.¹⁶ This process, reflective of a convergent parallel mixed-methods design, allows for a more complete understanding of the influence of BIE technology on the questions asked by preceptors.¹⁶

Participants and Setting

After institutional review board approval was obtained, I used purposeful and convenience sampling techniques¹⁸ to include a variety of participants and clinical settings. I recruited clinical education coordinators of Commission on Accreditation of Athletic Training Education–accredited athletic training programs within a 60-mile radius of my campus. After clinical education coordinators agreed to include their program in the research study, they provided contact information for preceptors and students within their athletic training program. I purposefully sought 1 : 1 and 1 : 2 preceptor-student groups in a variety of clinical practice settings to include diverse experiences and perspectives in the study. Clinical education coordinators, preceptors, and students were all required to complete informed consent forms before data collection was scheduled.

Recruitment resulted in the inclusion of 8 preceptor-student groups practicing in 7 different clinical sites affiliated with 3 undergraduate athletic training programs. The clinical settings included 3 secondary schools, 1 National Collegiate Athletic Association Division I university, 1 community college, 1 rehabilitation clinic, and 1 university club sports clinic. Eight preceptors and 13 students participated in the study. Pseudonyms were assigned to participants to protect their identities (Table 1).

Table 1.  Participant Demographics

View Fullscreen

Data Collection Procedures

In order to examine the questions asked by preceptors during clinical education experiences, I conducted field observations with audio recordings. I observed each preceptor-student group for 2 complete days of clinical education experiences without intervention. Observations started each day upon the student's arrival and concluded when the student departed for the day. Preceptor-student communications were audio recorded with lapel microphones (100-P Series, Sennheiser, Wedemark, Germany, and Pro 88W, Audio-Technica US, Stow, Ohio) transmitting wirelessly to an audio recorder (Zoom H2, Samson Technologies, Hauppage, NY) worn by the researcher. I observed and recorded field notes from approximately 5-to 6-m away to avoid interfering with participants' interactions.¹⁹

After the initial observations were completed, participants watched a 7-minute training video on the BIE technology created for this study, including technical instructions and suggestions for integrating the technology into clinical education. The video included sample patient-care scenarios and emphasized that preceptors should always be able to see and hear what their students are doing. Participants were provided the BIE technology and instructed to use it for 1 week of their normal interactions.

After participants used the technology for 1 week, the researcher returned and repeated 2 days of observations while the BIE technology was available to participants. Field notes were recorded when the technology was used for communication and questioning. Last, after all 4 days of observations were completed, I interviewed participants to obtain their perspectives of questions asked during clinical education experiences. Semistructured interview guides included general questions about preceptor-student interactions and specific questions related to questioning strategies of preceptors and the use of BIE technology (Table 2). Interview guides were peer-reviewed by a second qualitative researcher and piloted with 1 preceptor and 2 students before use in the study. Interviews were transcribed verbatim and provided to participants for member-checking. After participants member-checked their interviews, participation in the study was complete.

Table 2.  Interview Guides

View Fullscreen

Data Analysis and Trustworthiness

I listened to audio files and transcribed every conversation between a preceptor and student that included a question. These conversations were then linked to the field notes recorded during the observations to provide context to the conversations during the analysis process. Questioning statements were then coded by the researcher and a research assistant using the Question Classification Framework.^6,9,20 The framework categorizes low-level questions (information, knowledge, comprehension), high-level questions (application, analysis, synthesis, evaluation), and other types of questions (yes/no/recall, affective, rhetorical).^6,9,20 To establish consistency with coding, the 2 qualitative researchers independently analyzed 16 conversations (2 conversations from each group) on 3 different occasions. After each set of coding, we held a consensus meeting to discuss any differences and come to an agreement on our coding. Coders then independently coded every conversation, followed by 1 final consensus meeting to ensure they agreed upon all final coding. Last, an additional qualitative researcher peer reviewed 16 conversations, examining the codes in relation to the question classification framework. The peer reviewer confirmed the analysis, finalizing our questioning coding process.

Interview transcripts were analyzed by the researcher and an additional experienced qualitative researcher using a general inductive process of open, axial, and selective coding.¹⁷ Open coding included a general read and basic notation of participants' responses to interview questions.¹⁷ Axial coding included an organization of codes into categories that began to summarize multiple participants' thoughts. Last, categories with supporting codes were organized into themes that represented the majority of participants' responses.¹⁷

We independently coded 2 preceptor and 2 student interviews after this process to develop a draft codebook. We then analyzed remaining interviews with this codebook, meeting 3 times throughout the analysis process to add clarification and detail to the final codebook and agree upon final themes.

Multiple methods of trustworthiness were used throughout the data collection and analysis process to establish credibility of the research process. Peer review was used in the development of the interview guides and analysis of the questioning conversations.²¹ Participants had the opportunity to member-check their interviews, verifying their statements.²¹ Multiple qualitative researchers analyzed the questioning conversations and interviews.²¹ Data source triangulation occurred with the combination of researcher observations and preceptor and student interviews.²¹ Additionally, the variety of clinical settings and athletic training programs used in the study improve transferability of the findings.²¹ The previously used Question Classification Framework^6,9,20 also adds to the credibility of the analysis process.

To provide additional analysis of the questions asked by preceptors with and without the use of BIE technology, quantitative analysis of the coded questions was also performed. To provide descriptive statistics, the amount of time (minutes) and the average number of questions asked for each category was averaged over the 2 sessions with and without (control) the use of BIE technology. Although this qualitative design was not specifically powered for quantitative analysis, a 2 (Intervention) × 3 (Question Type) analysis of variance was performed to compare the number and types of questions asked between the control and BIE sessions. Post hoc t tests with Bonferroni correction were conducted in the event of an overall significant test.

Quantitative Findings

Preceptors asked a total of 1044 questions during 148 hours 53 minutes of observation. The cognitive level of questions asked by each preceptor with and without BIE technology are summarized in Table 3 and examples of each type of question are provided in Table 4. The average amount of time spent observing during the control and BIE sessions was similar (4 h 45 min ± 45 min versus 4 h 33 min ± 55 min; P = .69). Descriptive statistics for the average number of questions asked in each category are provided in Table 5 for the control and BIE sessions. Preceptors asked more questions during the control sessions than when they used BIE technology (39.1 ± 31.7 versus 26.1 ± 20.4 questions; F_1,7 = 6.3; P = .04).

Table 3.  Cognitive Level of Questions Asked by Each Preceptor With and Without Bug-in-Ear (BIE) Technology

View Fullscreen

Table 3.  Extended

View Fullscreen

Table 4.  Sample Questions Asked by Preceptors

View Fullscreen

Table 5.  Descriptive Statistics for the Amount of Time Observed and Average Number of Questions Asked During the Control (CON) and Bug-in-Ear (BIE) Sessions, Mean ± SD (95% Confidence Interval)

View Fullscreen

There was also a significant main effect for Question Type, but these data did not meet the assumption of sphericity, so the F-statistic was interpreted after applying the Greenhouse-Geisser adjustment to the degrees of freedom (F_1.2,14 = 10.2; P = .01).²² The post hoc pairwise comparisons of the question types indicated that a higher number of questions classified as “other” were asked compared with high-level questions (P < .01) regardless of whether BIE was used. There was a trend toward a greater number of low-level questions compared with high-level questions (P = .051). There was no significant Intervention × Question Type interaction (F_2,6 = 2.6; P = .16), indicating that there was no difference in the number of questions asked in some categories between interventions.

Qualitative Findings

In addition to qualitative coding and quantitative analysis of the actual questions asked by preceptors, interviews captured preceptors' and students' perspectives of questioning in clinical education. Analysis of the interview data revealed two themes: (1) Preceptors ask questions to develop clinical reasoning; and (2) BIE technology facilitated low-level questioning by preceptors. Both students and preceptors described the importance and actual delivery of questions to develop clinical reasoning, although few high-level questions were actually asked during observations. Aligned with quantitative analysis, participants perceived that BIE technology prompted preceptors to ask more low-level questions. However, participants thought BIE technology increased the number of questions asked, whereas the opposite was measured by the researcher. Themes and supporting quotes from participants are described below.

Cognitive Level of Questions

The findings of my study are consistent with previous studies^6,9,20 identifying that preceptors primarily ask low-level and basic yes/no questions of their students. High-level questions help develop critical thinking and problem solving, which are important components of clinical reasoning for health care professionals.²³ Therefore, it is concerning that only 2.38% of questions asked by preceptors targeted a high cognitive level. Researchers have suggested that high-level questions should be asked more often with advanced students.^7,8 All student participants in my study were enrolled in 3-year undergraduate athletic training programs, with 4 senior-level, 2 junior-level, and 7 sophomore-level athletic training students. Perhaps fewer high-level questions were asked in my study because most of the students participating were beginner or intermediate level.⁷ However, Barnum's⁶ study examining preceptors' questioning strategies included only 1 senior-level student, but more high-level questions (17.00%) were asked compared with my study (2.38%). Therefore, it appears that preceptors in my study asked fewer high-level questions than previously documented in the literature.^6,9

I found it interesting that despite the finding that preceptors asked so few high-level questions, participants spent much of their time discussing high-level questions in their interviews. Preceptors and students discussed the value of using questions to assess clinical reasoning and challenge students and provided several concrete examples from their current clinical experiences. These results are consistent with the findings of Mazerolle et al²⁴ that preceptors understand the importance of using questioning to facilitate student learning and recognize specific times when they used this strategy. My participants' comments were also reflective of suggestions in the literature regarding effective high-level questioning.^6–8

It is unknown why there was such a mismatch in what was observed versus what was discussed. Perhaps high-level questions make more of an impression or are more memorable, which is why participants spent so much time discussing them. It is also possible that the days observed did not accurately capture the high-level questions typically asked. Regardless of the differences between what was observed and what was discussed, it is promising that students and preceptors recognize the value of high-level questioning and the importance of developing clinical reasoning in athletic training students.²⁴

Although the level of questioning is thought to be important in developing clinical reasoning, several authors^6,8 have argued that the strategy and order of questioning may be more important than the cognitive level. Preceptors may ask several basic questions that lead toward more high-level questions to critically analyze a problem.^6,8 Additionally, some recommend that simple recall questions should be asked of students to assess their knowledge base before progressing to more complex questions.^6,8 Preceptors in my study asked many low-level (n = 557) and yes/no (n = 429) questions, resulting in 94.44% of the total questions asked. Although many of these questions targeted basic knowledge or information, many of them were used in sequence during a conversation and were not just asked in isolation. Preceptors and students also discussed purposeful sequencing of questions in their interviews, suggesting that many preceptors in my study also followed an intentional strategy to their questioning. For example, preceptor Anne said she begins by asking a student what they did with a patient, then why they chose to do so, followed by a discussion. This pattern of questioning mirrors a strategy used by Barnum and colleagues'⁷ and others concerning the “What, So What, Now What” process of strategic questioning that starts with basic knowledge and progresses to higher level clinical reasoning. Preceptors and students in my study described this strategic, sequenced pattern of questioning to be valuable for student learning. Therefore, I concur with Barnum⁶ that cognitive level is not the only way to assess quality of questioning by clinical preceptors—the sequence and context of questions should be considered.

Questioning and Bug-in-Ear Technology

My findings suggest that the use of BIE technology, or two-way radios, decreases the number of questions asked by preceptors. Additionally, I observed that no high-level questions were asked using the BIE technology. Participants also perceived that preceptors asked more low-level and fewer high-level questions when the technology was available. These findings suggest that BIE technology is not an effective strategy for increasing the quantity or quality of questions asked by preceptors during clinical education experiences.

My study is the first to examine preceptor use of BIE technology for questioning athletic training students. Additionally, researchers in other fields have yet to specifically investigate questioning and BIE technology. Existing research has focused on the application of BIE technology to improve feedback and communication, which has produced positive results.^12–14 Researchers in teacher education^12,13 and physical education¹⁴ have found increases in communication, feedback, autonomy, and satisfaction using the BIE technology compared with these educational experiences without the technology. Students and teachers perceive that BIE technology facilitates discussion and makes communication easier and more efficient,^13,15 including discussions about decision-making.¹⁴ Although most participants described that more questions were asked with the BIE technology, preceptors asked significantly fewer questions on the days they had the radios (n = 418) compared with days they did not have the radios (n = 626). The contrast between what was observed and perceived could be attributed to several reasons. Although not statistically significant, the average control day was longer than the average BIE day, which decreased the time available for preceptors to ask questions. Additionally, participants used the BIE technology for 1 week before the researcher resumed observations; therefore, more questions may have been asked on those days than the observation days. The differences may also be due to a misperception by participants, which has been noted in other research comparing perceived versus actual behaviors.^25,26

It is unknown why BIE technology did not improve questioning in my study considering that the technology improves communication in other educational settings.^12–14 It is possible that the athletic training clinical setting is different enough from teacher and physical education experiential learning environments to limit its applicability to this setting. Teacher educators may be able to focus more on a student teacher's performance with less multitasking than preceptors, which may allow teacher educators to more closely supervise and communicate with their students. Preceptor role strain and challenges with role balancing have been well documented in the literature^11,27,28; thus, if preceptors are often occupied with their own patients it may be challenging to ask students high-level questions at the same time.

It is also possible that questioning, particularly high-level questioning, is a different type of communication strategy better conducted in person rather than over a radio, as one of the preceptors in my study commented. Only 11.72% of the questions asked on the BIE-technology observation days were asked using the technology, so participants may not have used the radios much for any communication, including questioning. Athletic training researchers^7,8 have suggested that questions should be adapted as the clinical experience progresses. Thus, the 2-week delay between regular observations and observations with the BIE technology may have influenced the type and quantity of questions asked, regardless of the availability of BIE technology. Clinical education coordinators and preceptors should consider the potential positive and negative impacts of using BIE technology before implementing this strategy in clinical education.

Limitations and Future Research

My study examined BIE technology and questioning with a small number of participants within only 3 undergraduate athletic training programs; therefore, findings may not extend to professional master's students or a broader population of preceptors and students. Observations were completed 4 days for each group of participants at different time points throughout 1 semester. Lengths of observation days were intended to capture each student's experience, start to finish, so observation lengths were not standardized. Thus, the observation days may have differed between participant groups and may not have captured their typical interactions. Similarly, presence of the researcher or participation in the research study may have influenced their behaviors. Researchers may investigate this topic further by including a larger or more diverse group of participants over a longer period of time to improve transferability of the findings.²¹ Also, researchers may consider an in-depth case study approach¹⁷ with more observations to understand how questioning and BIE technology may change throughout a clinical experience or over multiple clinical rotations.

Conclusions and Implications

The findings of my study are consistent with previous research demonstrating that preceptors rarely ask high-level questions during clinical education experiences.^6,9 However, participants spoke extensively about high-level questions used by preceptors to facilitate clinical reasoning, reinforcing that asking high-level questions as a means to develop clinical reasoning is valuable and occurring during clinical education.^6,8,24 The rare use of high-level questions in my study suggests preceptors should be taught how to integrate more high-level questions with their students. However, strategic questioning and effectively sequencing questions should be reinforced in addition to the cognitive level of questions asked.^6,8 Preceptors should be encouraged to follow low-level questions with high-level questions that challenge students' thought processes.⁷ Preceptors may also need more education and guidance on the provision of effective questioning, which can be provided through professional development sessions, preceptor workshops, and mentoring. Clinical education coordinators should consider integrating instruction on effective questioning strategies in preceptor workshops and communications with their preceptors.^7,8

Although BIE technology has been successfully integrated into other experiential learning environments to improve communication and feedback,^12–14 the use of the technology led to a reduction in the quantity of questions asked in the athletic training clinical education setting. These findings may be attributed to the different type of learning environment, instructional strategy, implementation method, or methods of observation. Thus, the use of BIE technology and questioning in athletic training should be investigated further. Participants in this study perceived that BIE technology did make communication, including asking quick, basic questions, easier. Clinical education coordinators and preceptors may try BIE technology as a clinical instructional tool to facilitate communication or for other reasons described in the literature.^12–14 If so, users should be encouraged to demonstrate established effective preceptor characteristics, such as supervision, communication, questioning, and feedback^7,29 while using the technology.

An interesting finding of this study is the discrepancy between participants' perceptions and actual questions asked during clinical education. Preceptors and students provided detailed examples and explanations of the value of asking questions that promote clinical reasoning, yet few of these questions were actually asked during observed days. Additionally, most participants perceived that preceptors asked more questions with the BIE technology; however, they asked significantly fewer questions on the days the technology was available. Differences between student, preceptor, and observed behaviors has also been noted in existing research.^26,30 This finding suggests that clinical education coordinators should evaluate clinical experiences from multiple perspectives, including preceptors, students, and outside observers (eg, clinical education coordinator site visits). Research studies should also attempt to measure study outcomes from multiple perspectives to facilitate thorough and accurate examination of the topic of interest.