The Relationship Between Candidate Psychological Factors and First-Attempt Pass Rate on the Board of Certification Examination

INTRODUCTION

Certification of athletic trainers by the National Athletic Trainers' Association began in 1969. Since 1989, the Board of Certification, Inc (BOC) has been the autonomous “nonprofit credentialing agency that provides certification for the athletic training profession.”^1(p3) The BOC's stated mission is “to provide exceptional credentialing programs for healthcare professionals to assure protection of the public.”^2(p5) The Certified Athletic Trainer (ATC) credential is the cornerstone for regulation in many states, and the BOC certification exam meets requirements in 47 of the 48 states that regulate athletic trainers.³ In order to obtain entry-level certification, “an individual must graduate from an athletic training education program accredited by the Commission on Accreditation of Athletic Training Education (CAATE) with a Bachelor's or Master's degree and pass the BOC certification exam.”^2(p5)

Beginning in June 2007, the national certification examination moved from a traditional paper-and-pencil examination to one that was completely computer based. The exam has 175 scored and unscored multiple-choice questions, stand-alone alternative items (including drag-and-drop, text-based simulation, multi-select, hot spot), and focused testlets (consisting of a scenario and 5 key/critical questions related to that scenario).^1,2 These components assess whether the candidate sufficiently understands the principles, practices, and sciences underlying the practice of athletic training as described in the Role Delineation Study/Practice Analysis, 6th edition. The test also measures a candidate's decision-making skills and whether they demonstrate entry-level proficiency in the skills required for employment in a wide variety of settings.²

Historically, a concern with this examination has been the student first-attempt pass rate. According to the 2008 BOC Annual Report,⁴ the percentage of candidates passing the entire examination on the first attempt in 2007–2008 was 39.1%. This is well below what many in the profession would expect, given that it reflects poorly on the educational institutions these students attended. However, this pass rate was based on the paper-and-pencil examination. The first-attempt pass rate in 2010–2011 on the computerized exam improved dramatically¹ to 82.3%. Although this was an improvement, having nearly 18% of the eligible candidates fail the exam on the first attempt is not the preferred outcome for those graduates of CAATE-accredited athletic training programs who did not pass on the first attempt.

The testing process is stressful, and students may believe that not passing the exam will make it difficult to obtain employment, a graduate assistantship, or an internship position. Unfortunately, this self-imposed pressure and the resulting anxiety can negatively impact their test-taking ability. Therefore, it is proposed that students' academic worry, locus of control, and method of coping (emotion-focused versus problem-focused) factor into their success on the BOC examination.

Most recent studies related to the BOC examination have focused on its relationship to a student's type of clinical experience, the number of clinical hours earned, and his or her grade point average.^5–9 Successful performance is often the product of complex relationships between skill and knowledge, mediated by perceptions of anxiety, self-confidence, and preparedness.¹⁰ To date, there have been no published studies examining anxiety/academic worry, methods of coping, and locus of control in athletic training students, but there is literature on both general test taking and medical students' performances on standardized tests.^10–15

METHODS

RESULTS

Correlation coefficients showing the relationships among the scores for Academic Worry, Locus of Control, Emotion Focus, and Problem Focus are shown in Table 1. Increased emotional focus correlated to decreased (more internal) locus of control (P = .001) and increased academic worry scores (P = .00). Similarly, an increased academic worry score correlated to increased emotional focus (P = .00) and a decreased (more internal) locus of control (P = .006). However, increased (more external) locus of control correlated to decreased emotional focus (P = .001) and decreased academic worry (P = .006). There were no significant correlations found between increased problem focus and the other scores.

Table 1.  Correlations (N = 145)

View Fullscreen

Although both the Locus of Control (α = 0.71) and Academic Worry (α = 0.79) scales showed adequate internal consistency, this calculation was not run for Emotion-Focused Coping and Problem-Focused Coping because those scales measured the frequency of related behaviors that are not necessarily consistent with one another.

The results of the logistic regression analysis suggested that the full model significantly predicted passing on the first attempt ( = 15.346, P = .004). The model accounted for 10% to 15% of the variance. Overall, 96.2% of the participants who passed on the first attempt and 25.6% of those who did not pass on the first attempt were correctly classified (77.2% of the total participants were correctly classified). It is likely that the lower explanatory power of the “did not pass on first attempt” group is due to the small sample size of that group. As shown in Table 2, the values of the coefficients indicated that a 1-point increase in Locus of Control score was associated with increased odds of passing on the first attempt (adjusted odds ratio = 1.033). No other covariate was found to be significant.

Table 2.  Logistic Regression Analysis

View Fullscreen

The results of the t tests (Table 3) indicated that, compared with those who passed the test on the first try, those who did not pass on the first attempt had significantly lower scores on the Emotion-Focused Coping scale (P = .024). There were no significant differences on the Problem-Focused Coping scale between those who passed the test on attempts after the first and those who had not yet passed the test. Frequency of responses to individual items, subdivided by emotion focused and problem focused on the Ways of Coping scale, are shown in Table 4. In keeping with the original authors' article, item 11 was not included on either of the scales.²⁴ However, those who passed on the first attempt did have a higher mean score on several study variable scales than those who did not pass on the first attempt (Table 3). When compared with those who passed the test on the first attempt, those who did not pass on the first attempt exhibited more external locus of control (P = .002) and greater academic worry (P = .026).

Table 3.  Descriptive Statistics

View Fullscreen

Table 4.  Frequency Table–Coping

View Fullscreen

Although not statistically significant, the t test means approached significance for participants who answered no to the questions “Did teaching faculty prepare you well?” and “Did clinical instructors prepare you well?” (Table 5). Results of the t tests also indicated no difference in the number of prep items used between those who passed on the first attempt and those who did not pass on the first attempt. However, the data also suggested a normal distribution for the total number of types of exam prep techniques used by both first-attempt passers and the others, with the majority of both groups having used two prepping techniques.

Table 5.  Preparation by Faculty

View Fullscreen

DISCUSSION

The goal of this exploratory study was to understand the relationship between locus of control, method of coping, academic worry, and first-attempt passing rates on the BOC examination. Unfortunately, the participants who did not pass the BOC exam on the first attempt were underrepresented in the sample. Due to that fact, the different groups may or may not represent the population at large, and the findings should be taken as preliminary results. Replication of this study with a larger randomized sample would help to confirm these results.

The findings suggest that academic worry and emotion-focused coping methods may have some effect on those who do not pass the exam on the first attempt. These data are consistent with findings by Ames,¹⁶ Struthers et al,¹¹ and Weiner¹⁷ that worry about various academic experiences perceived as negative could have an adverse effect on student performance. It appears that the results also show that those students who used more emotion-focused coping methods to deal with situational stress were more likely to fail the examination, whereas those who used a more problem-focused coping strategy were more successful. Building self-efficacy through BOC examination rehearsal and reinforcing positive coping behaviors may effect success on the first attempt. The results also showed that students who did not pass on the first attempt felt that academic and clinical faculty could have done more to prepare them for the BOC examination (Table 5).

Although previous studies indicated that a more internal locus of control was associated with academic success,³⁴ our results did not replicate them. On the contrary, respondents with a more external locus of control tended to pass on the first attempt at a higher rate than those with an internal locus of control. This may be due to either the small sample size or additional, unexplored individual differences within our self-selected sample.

This study begins to show the basis for a strategy to help students prepare for success on the BOC examination, similar to what is in place in nursing education. That program, known as KATTS, addresses the following: knowledge base, anxiety control, and test-taking skills. It is a strategy that is program based and has improved NCLEX-RN success from 85% in 2000, the year before KATTS was initiated, to 97% from 2001–2006 when the program was put into place. The KATTS program uses personal wellness strategies, individualized review techniques, and practice/rehearsal routines in a positive context to help prepare nursing students for the NCLEX-RN.¹⁴

An example of this approach in athletic training would embed this strategy into a seminar class taken in a program's last semester to prepare students for either the November or April BOC examination dates. Self-efficacy and academic worry are addressed by identifying their knowledge base, including perceived student strengths and weaknesses. This process begins in the first week of the semester when students complete a “confidence survey” based on the 6th edition of Athletic Training Role Delineation Study and Practice Analysis. This document provides the basis for the items in the BOC examination. The students then complete a 100-question quiz from an exam preparation text, and the results are compared with the confidence surveys. This provides an explicit foundation for addressing individual strengths and weaknesses in the exam preparation process.

Test anxiety and test-taking strategies on computerized standardized examinations are also addressed through rehearsal. Approximately 1 week after the initial assessment of strengths and weaknesses, the students take a mock exam using a practice exam purchased from the BOC. These exams take place in a computer lab to simulate the actual testing process. The students review their results with the instructor for additional study strategies. Students usually take a second mock exam later in the semester, after spring break, and generally perform much better. It is felt that this process, similar to what has been done by nursing educators, decreases test-taking anxiety, increases self-efficacy regarding competence in athletic training, and empowers students to take an active role in BOC examination preparation.^14,23 The focus is neither for students to “cram” nor for instructors to “teach to the test,” but rather to provide the students the cognitive and emotional feedback necessary to give them confidence when taking this important high-stakes examination.