Exploration of Factors Perceived to Influence Development of Diagnostic Reasoning in Athletic Trainers and Athletic Training Students

Participant Selection

All AT and athletic training student participants were originally recruited for a larger, multifaceted diagnostic reasoning study involving an objective measurement of diagnostic thinking. Athletic trainers were recruited by sending a participation e-mail to a randomized list of names and e-mail addresses obtained from the National Athletic Trainers' Association. Athletic training students were recruited through e-mails sent to random and purposefully sampled athletic training program directors, asking them to forward the participation e-mail to each senior-level athletic training student in their program. Additionally, senior-level athletic training students were recruited in person at the 2015 Southeast Athletic Trainers' Association Athletic Training Student Symposium to participate in the study. The results of the larger study are not presented in this paper. Originally, 103 randomly sampled professional and student participants completed the Diagnostic Thinking Inventory for Athletic Training (DTI-AT)¹⁸ at a site and time of their choosing. Participants were contacted by e-mail within 6 hours after completion of the DTI-AT and asked to participate in an online interview, using either Skype (Microsoft, Redmond, WA) or Apple FaceTime (Cupertino, CA). If a participant agreed to the interview, it was scheduled to take place within 48 hours after completion of the DTI-AT. The purpose of scheduling interviews within close proximity to DTI-AT completion was to allow for clinical reasoning processes to be fresh in the participants' minds, potentially leading to more accurate reflections of clinical reasoning experiences. Because data analysis occurred simultaneously with data collection, interviews were conducted until data saturation occurred, at which point we ceased contacting participants. Thirty participants were contacted and 23 were interviewed (11 ATs, 12 athletic training students). Participants consisted of 14 women (61%) and 9 men (39%) with the following education levels: 12 with a high school diploma (52%), 3 with a bachelor's degree (13%), 7 with a master's degree (30%), and 1 with a doctoral degree (5%).

Inclusion criteria stated each AT had to be currently practicing or educating, and each athletic training student had to be over the age of 18, in the senior year of an athletic training program curriculum, and considered a full-time student in an undergraduate professional education program. Exclusion criteria included professional graduate program students and any AT who had maintained certification status but was not currently working in a clinical or educational capacity within 6 months of the study.

Instrumentation

Experts in qualitative research design who were familiar with diagnostic reasoning theory constructed the interview template used for data collection. The interview questions and procedures were piloted by using senior athletic training students from the investigators' institutions; they were interviewed and queried in a follow-up dialogue to ensure that all questions were clearly understood. Pilot data from the interview were analyzed using a general inductive approach and critical-friend analysis, and it was determined that the information gathered coincided with the interview purpose.¹⁹ Based on feedback in the pilot process, no major question revisions were deemed necessary, but examples of potential query items to enhance or hinder diagnostic reasoning were added to provide context for the participants. The interview questions were as follows:

1. When confronted with a diagnostic opportunity, please tell me how you reason through your evaluation process. In other words, please describe your thought process when making a clinical decision.

2. Please explain some personal activities you have engaged in over time you feel have helped to develop your diagnostic reasoning ability.

3. Please explain some professional activities (clinical experiences and professional development for students) you have engaged in over time you feel have helped to develop your diagnostic reasoning ability.

4. What are some issues in your personal life that act as barriers to your diagnostic reasoning development?

5. What are some issues in your professional life (clinical experiences and professional development for students) that act as barriers to your diagnostic reasoning development?

All interviews were conducted through Skype or Apple FaceTime, with the exception of 1 participant who was interviewed over the phone because of technical issues that could not be rectified. Before participating, participants were required to read and sign an electronic or paper copy of the informed-consent document. All interviews were digitally recorded and both the interviews and transcriptions were stored on a password-protected hard drive. The principal investigator and 1 other individual not associated with the study transcribed all interview data. Once accurate transcription was ensured, pseudonyms were provided for the participants to ensure confidentiality and digital copies of the interviews were deleted. This study was approved through institutional review board before data collection initiation.

Data Analysis

We used a general inductive approach to analyze the interviews with the intent of determining participants' understanding of their diagnostic reasoning processes as a whole, as well as which personal and professional activities they perceived to enhance or hinder their diagnostic reasoning ability.¹⁹ The general inductive approach uses an analysis methodology similar to grounded theory, but is more appropriate for condensing extensive data and providing links between themes rather than for developing one specific theory,¹⁹ an approach documented as viable for analyzing qualitative data.^20,21 Data were analyzed separately for the AT and athletic training student participant groups in order to assess differences in perceptions between groups. The transcribed interviews were first analyzed by the principal investigator independently in order to identify and label specific segments of words or phrases from each response. The overlapping information was then compared and combined to form subcategories based on our stated research objectives, and associated participant quotes were assigned to the subcategories.¹⁹ The subcategories were extrapolated into larger themes once saturation was determined using the constant-comparative approach.¹⁹ Saturation was considered reached by the principal investigator during the formation of subcategories when new information that might lead to the formation of a new subcategory or theme was no longer apparent. After finalization of themes and subthemes, a coding consistency check was performed by having an independent researcher who has published experimental clinical reasoning articles demonstrating expertise with diagnostic reasoning theory assess the coded data to help reduce potential research bias, challenge coding assumptions, refine operational terminology, establish connections between the data analyzed and other research findings, and confirm data saturation.¹⁹ A literature consistency check was then performed by having the 4 coauthors compare the exposed themes with research in other fields and provide feedback for clarity between themes and associated participant quotes.¹⁹ Independent analysis, coding consistency check, and coauthor literature consistency check techniques were considered sufficient to ensure trustworthiness of the data collected. Of the 5 interview questions, the first was designed to evaluate participants' understanding of their own diagnostic reasoning thought process, questions 2 and 3 were intended to weigh practices perceived to enhance their diagnostic reasoning, and questions 4 and 5 were specifically designed to reveal any practices perceived to act as barriers to diagnostic reasoning development; therefore, the themes and subthemes were separated into these 3 categories.

Comparing ATs' and Athletic Training Students' Thought Processes

The purpose of our study was to conduct a preliminary investigation into ATs' and undergraduate athletic training students' perceptions about their diagnostic reasoning processes and experiences. Secondarily, we aimed to identify and compare activities or practices that may enhance or hinder individual diagnostic reasoning abilities. The first interview question was designed to determine participants' understanding of their cognitive processes during a diagnostic encounter. The interviews occurred within 48 hours after each participant completed the recently validated DTI-AT by Kicklighter et al,¹⁸ in which participants encounter a common orthopaedic diagnostic scenario and then are asked to reflect on their diagnostic reasoning over 41 questions, or cues.¹⁸ Our intent was for this metacognitive activity to provide participants with a reference point from which to address our first question with more introspection and personal analysis. Given that our respondents were then able to make specific comments about their didactic and clinical experiences, to provide specific examples, and to articulate many of the foundational underpinnings of metacognition, reflection, and reasoning found in both practice and the literature, we are confident that our goals were met and that operationally, our methods were effective for addressing our research questions.

To have a standard reference for comparison, it is necessary to determine if the processes presented within our results coincide with established theory of experts and novice diagnostic reasoning processes found in other health care fields. Recent research by Gardin and Mensch¹⁶ in athletic training has begun to demarcate expert diagnostic reasoning through experimental and qualitative research.¹⁷ However, the complex nature of diagnostic reasoning expertise requires further research before we are able to truly label an individual as an expert or novice, especially in athletic training contexts, as the topic has largely been ignored to date. For the purpose of this study, the terms expert and novice are used based on professional versus student experience, respectively, and therefore used to compare our findings of participants' perceptions with established characteristics found in current diagnostic reasoning research. As the authors, we are not attempting to firmly objectify or define the specific characteristics of experts or nonexperts in athletic training. Given this limitation, it is well reported in the medical literature that experts take a minimum of 10 years of deliberate practice, and in doing so develop, display, or possess, on some level or another, more expert diagnostic reasoning (CPR) abilities, an authentic commitment to lifelong learning, and a habit for meaningful and structured reflection and metacognition.¹⁵ Though we did not set out to define expertise in athletic training, it is clear from the information we collected that these thematic characteristics of medical expertise are indeed operational in some format, and on some level in athletic training. Much more research needs to be carried out on the idea of expertise in athletic training, research designed to address such questions as, for example, “What is expertise in athletic training?”, “What is an expert AT?”, and “How is expertise developed or built?”.

In our study, the AT participants demonstrated some of the known characteristics of expert-level diagnostic reasoning by describing how they quickly and accurately form and test a working diagnosis via very complex and interrelated mechanisms. Consistent with findings from medical literature, experienced ATs in our study reflected more expert behaviors such as recognizing key features and unique features that don't fit,⁴ forming plausible yet small differential diagnosis lists, relying on bioscientific knowledge, reflecting on past case experiences, and subconsciously using semantic axes to decipher and connect patient-provided information into appropriate medical contexts and clinical meanings. Semantic axes are defined as cognitive connections between various presenting signs or symptoms and actual diagnoses that exist in a clinician's experienced and organized mind.²² This neurological scaffolding helps more expert clinicians prioritize and recognize existing relationships among previously learned information or “key features” to recall known clinical case patterns.^22,23 Our results demonstrate that expert-level, subconscious cognitive reasoning in ATs occurs during the history and observation portions, which is consistent with prior findings that have reported that 80% to 90% of diagnoses are formulated early in the subjective portion of the evaluative process.^24,25

Additionally, our diagnostic reasoning findings align with those reported by Cappelletti et al²⁶ in their systematic review of the nursing literature regarding expert thinking. Cappelletti and colleagues²⁶ demonstrated experts convey preestablished, organized memory structures along with a flexibility of cognitive strategies when encountering more difficult clinical situations, which is similar to Norman's² contemporary dual processing theory for diagnostic reasoning. Dual processing posits that although experts primarily use nonanalytic CPR-based approaches to diagnostic reasoning in familiar cases, they must also think flexibly by using HDR in the presence of unusual information or atypical cases in order to avoid overlooking important signs and symptoms and making a diagnostic error.^5,27 These findings coincide with Gardin and Mensch's¹⁶ findings that athletic training experts use more than 1 type of thought process when making diagnostic decisions, depending upon the context.

Responses to our first line of student questioning indicate our athletic training students use a more extensive, analytic, diagnostic methodology consisting of excessive data collection, limited script development, and a lack of confidence in the eventual diagnosis. This analytic HDR processing is appropriate for and expected of novices, and should thus be encouraged in all student novices as the preferred mode of thinking for initial clinical encounters, as less-experienced clinicians require safety in and find comfort with greater structure and comprehensiveness. However, it must also be cautioned that an exclusive use of HDR is associated with increased diagnostic error because of the limited ability of the novice mind to process and prioritize relationships among the large amounts of data (signs and symptoms, exam findings, etc) effectively. In other words, cognitive overload can inhibit the ability of the investigating mind to filter appropriate from inappropriate information, and it can overtax the short-term memory capacity of even the brightest and best students. Because of its necessity and students' propensity to organize and understand accumulated data with new cases, HDR is an expected and necessary entry-level cognitive process for professional students, and thus should be mastered before progressing to more advanced CPR approaches. As knowledge acquisition and experience with clinical cases mount, CPR strategies can and do evolve organically, but there is also a personal and experiential context to consider, as each student will waver or float between HDR and CPR mechanisms depending on his or her relative knowledge, experiential encounters, and metacognitive acumen.⁴ Nevertheless, it is indeed interesting to note that the level and type of diagnostic reasoning reported by our undergraduate athletic training participants is similar to and consistent with that found in first-year medical students, despite the obvious differences in age and educational background.^28–30

Regardless of what type of reasoning a particular student or practitioner employs or describes, the potential factors perceived to enhance or hinder diagnostic reasoning must be assessed in order to better appreciate diagnostic reasoning ability and to determine the validity of participant perceptions.³¹ Therefore, it was prudent for us to explore the perceptions our participants had regarding potential factors or experiences that influenced their diagnostic reasoning abilities in order to help future investigations address them in isolation and context.

Collective Perceived Enhancers of Diagnostic Reasoning

In both groups, the most important factor perceived to impact current state of diagnostic reasoning was professional socialization. This collaborative and interactive dynamic appears to provide students and young clinicians with multiple perspectives for thinking, possibly leading to increased structure of memory through continual feedback, script development and reformation, and better cognitive organization. We also found that diversifying one's initial skill set, using mentors for feedback and challenge, and active participation in professional conferences are other ways to increase thinking and knowing perspectives. Although this was not directly stated by our participants, previous research has demonstrated that professional socialization is considered a form of metacognition, given the fact that alternative perspectives require reflection and analysis of current abilities and understanding, possibly refining and enhancing diagnostic abilities and thinking processes.^26,32 Our findings are further validated when compared with the findings of Gardin and Mensch¹⁶ that varied experiences combined with consistent internal and external feedback are self-regulatory metacognitive activities considered to enhance diagnostic reasoning development.¹⁷ For our student participants, educational and professional socialization need to incorporate structured, autonomous, varied patient encounters emphasizing the transfer and integration of previous and relevant bioscientific knowledge, as well as use of effective and meaningful semantic axes in order for script or case pattern development, and these activities and challenges clearly need to occur in classroom and clinic alike.^4,6,33

Using physicians and interns, Durning et al³² provided a qualitative analysis similar to that of our study in order to determine what activities may develop and maintain diagnostic reasoning in different stages of a physician's career. Durning et al³² similarly determined specific undergraduate and graduate teaching strategies (both didactic and clinical), patient encounters, teaching others, having a mentor, and self-directed learning to be the most important factors for developing and maintaining diagnostic reasoning abilities. These findings align with our ATs' and athletic training students' findings regarding the importance of specific educational strategies and professional socialization in the development of clinical competence and professional transition.^34,35 It was interesting to note, however, that neither the AT population in our study nor the physicians used by Durning et al³² expressly determined reflective or metacognitive activities as being perceived to enhance diagnostic reasoning, despite the considerable research stating otherwise.^26,33,36,37 Most current research in diagnostic reasoning development states that metacognitive activities are imperative because they help to refine judgments, prioritize relationships among signs and symptoms, and force individuals to evaluate the cognitive processes used during a diagnosis.³⁶ Interestingly, our results indicate that student respondents favored self-directed learning as a metacognitive strategy rather than the traditional sense of reflection, a finding that aligns with current research into the development of diagnostic reasoning.^33,36,38

In fact, our athletic training student population seemed to possess a deeper grasp of at least some of the concepts considered to enhance and hinder diagnostic reasoning than did our professional participants.^13,39 This curious finding may be at least partially attributable to the notion that current athletic training students are being more formally exposed to diagnostic reasoning development strategies in their didactic and clinical experiences, and perhaps even have more thinking-focused clinical education experiences compared with practicing ATs, who may have finished their formal education some time ago. To that end, recent exposure to diagnostic reasoning–related theory, application, and research at athletic training education conferences and journals, and advances in required educational standards, including the Commission on Accreditation of Athletic Training Education competency requirement for diagnostic reasoning–based education and formalized preceptor training, may suggest that the future AT population should be engaging in more structured metacognitive activities to enhance all facets of diagnostic reasoning.

Collective Perceived Barriers to Diagnostic Reasoning

Cappelletti et al²⁶ determined that social factors associated with work and education environments play large roles in either enhancing or hindering diagnostic reasoning development. Lack of or ineffective professional and educational socialization and perceived or real time constraints were 2 of the largest barriers to diagnostic reasoning development in both groups of our current study. In our AT population, a lack of professional collaboration and debriefing was perceived to limit the ability to gain multiple diagnostic perspectives, which may in turn limit future script development and refinement, self-reflection, and even future memory organization. Additionally, time constraints may force individuals to rush through diagnostic challenges, potentially overlooking important details and limiting the flexibility of thinking (not to mention dangerous and inaccurate outcomes). These issues are certainly not specific to athletic training, and have previously been noted as meaningful barriers to diagnostic reasoning in other health care fields.^26,40–43 On another level, however, this finding is somewhat counterintuitive to some aspects of diagnostic reasoning theory, because expert-level diagnostic reasoning ability ultimately requires less time to more accurately diagnose an illness or injury. Therefore, perceived time constraints should, in fact and in the minds of capable clinicians, actually promote the use of CPR by forcing individuals to refine their cognitive strategies, leading to more efficient diagnoses. Or alternatively, perhaps assessment of other metrics of clinical expertise would reveal that more expert thinkers actually operate better under tighter time constraints. Combined with ineffective professional transfer and perceived job pressures due to lack of resources and staffing, it is perhaps understandable that less-experienced clinicians would consider time as a barrier to diagnostic reasoning despite the logical application of diagnostic reasoning theory, which states otherwise.

As expected, the barriers that our students perceived were consistent with studies across other health care fields and focused mainly on a lack of time, educational frustration, and various personality attributes. The perception that there is too little time to think is very similar to that of their professional counterparts, with students feeling as though their schedule, personal life/family commitments, or lack of good time management skills prevents them from studying further to improve their diagnostic reasoning ability. However, this logic can be viewed as faulty, because research has shown that the gaining of knowledge does not directly lead to increased diagnostic reasoning as much as understanding and use of specific cognitive processes.^40,44 Time constraints are unavoidable in any allied health care education because of the dense structure of course sequencing and intense clinical rotation requirements; therefore, it is imperative that educational experiences be designed to lead students towards a less time-intensive, more nonanalytic approach to thinking by using specific diagnostic reasoning pedagogy, ultimately enhancing students' and professionals' ability.

Perceived ineffective preceptor and professor interaction was a large and, to be candid, disturbing theme of this study, and has been well documented as a barrier not only to diagnostic reasoning, but to clinical education in general.^45–51 This barrier can stem from clinical supervision from less-experienced preceptors who have not yet developed expert diagnostic reasoning,⁵¹ increased preceptor role strain,⁵² and lack of formal clinical instruction strategies.⁴⁶ Just recently, Geisler⁵³ published a concise, evidence-informed piece for developing diagnostic reasoning abilities in students and young clinicians alike, and, interestingly enough, the piece both harmonizes with and summarizes the findings of this investigation as it highlights the various clinical and didactic approaches that help develop CPR skills for diagnostic challenges. Along with this preceptor frustration, a professor not instructing in a way that is perceived to enhance diagnostic skills was seen as a large barrier. However, it is unclear whether the instruction is truly ineffective or just perceived as ineffective because of conflicts in learning and teaching styles. Regardless, it is imperative that more research into clinical and didactic education methods intended to enhance diagnostic reasoning now be validated using current qualitative methods such as the DTI-AT in order for more effective educational interventions to be documented.³¹ In a thorough review recently published in Medical Education, Schmidt and Mamede⁵⁴ outlines a 3-stage model for teaching diagnostic reasoning in both the classroom and the clinic. Stage 1 entails detailed causal knowledge of disease and illness through pathophysiological and pathomechanical principles (which, of course, are first founded upon other basic sciences like anatomy and physiology); stage 2 uses pathophysiological knowledge encapsulated into diagnostic labels by introducing the transfer and applicability of information learned in stage 1 (and prior); and stage 3 is founded upon the development and learning of rudimentary illness/injury scripts, whereby key features and cues are used to foster recognition of typical case patterns.⁵⁴ Interlaced with this, of course, would be an intentional curriculum designed to reflect a parallel, interrelated, reflective, evidence-informed, and deliberate practice-based clinical education component.⁵⁴

Lastly, common and challenging athletic training student personality traits such as lack of initiative, low confidence, and poor motivation to engage in more diagnostic opportunities were thought to hinder diagnostic reasoning advancement. Although these have been recognized in previous athletic training and other health care studies as common traits among students,^55,56 providing more effective and efficient diagnostic reasoning–based education should increase confidence and initiative in students as they progress through a well-structured AT curriculum and gain meaningful experiences from their intentional clinical education program.⁵⁴ Most experts agree that using specific methods to enhance diagnostic reasoning may increase students' confidence by increasing their ability to easily and correctly solve diagnosis, rehabilitation, or treatment problems, leading to increased self-efficacy and positive reinforcement.^4,5,12,33,53