Practice-Based Research Networks, Part II: A Descriptive Analysis of the Athletic Training Practice-Based Research Network in the Secondary School Setting

Design

Both prospective and retrospective designs were used to perform a descriptive analysis of the AT-PBRN. A prospective survey was used to obtain data regarding the participating secondary school clinicians and their CPS characteristics. A retrospective analysis of patient records within a Web-based electronic medical record (EMR) used by all clinicians was evaluated for patient and practice characteristics.

Participants

Participants included clinicians at secondary school sites participating in the AT-PBRN and the patients whom they evaluated and treated between September 1, 2009, and April 1, 2011. The study was approved by the A.T. Still University Institutional Review Board. Patient data obtained from the EMR were deemed exempt by the Board due to a Certification of Honest Broker System/Processes provided by the EMR developer, Essentialtalk (Calgary, AB, Canada), to the AT-PBRN research team. This agreement details the provision of a limited dataset from the EMR to the research team that is stripped of all federally defined personal identifiers (ie, protected health information).¹⁶

Instrumentation

Procedures

Data Acquisition

Statistical Analysis

Data were analyzed using descriptive statistics (percentages and frequencies) for all variables of interest. The variables analyzed in this study were classified into 4 groups: CPS characteristics (location, school type, enrollment, number of sports, number of athletes), clinician characteristics (age, sex, years certified, level of education, employment model, years at CPS, supervisor, average work hours, teaching duties), patient characteristics (age, sex, grade level, sport, reason for visit, initial evaluation diagnosis), and practice characteristics (treatments provided at initial evaluation, treatments provided by diagnosis, treatments as a function of reason for the visit).

Secondary School CPS Characteristics

Secondary school clinicians within the AT-PBRN practiced in 23 distinct CPSs across 7 states (Arizona, Connecticut, Massachusetts, Minnesota, New Hampshire, Vermont, Virginia). Most CPSs were associated with a regional coordinator (athletic training academic program = 13, clinic center = 2, hospital center = 5), with the remaining 3 CPSs acting as independent sites.

Most secondary school CPSs were located in public or public charter schools (n = 19, 82.6%), with the remainder in private parochial or private charter schools (n = 4, 7.4%). Most CPSs had a school enrollment of more than 1000 (n = 14, 60.9%). The numbers of male and female athletes at the CPSs are shown in Table 1. At the time of the study, the AT-PBRN captured 17 male and 20 female sports. The most common male and female sports were soccer (n = 22, 95.7%) and softball (n = 22, 95.7%), respectively (Table 2). The most common methods of signing on to the CORE-AT EMR were through a laptop or netbook computer (n = 15, 65.2%) and wireless or broadband mobile card (n = 14, 60.9%).

Table 1.  Number of Athletes at Participating Clinical Practice Sites

View Fullscreen

Table 2. Male and Female Sports Captured Within the Athletic Training Practice-Based Research Network^a

View Fullscreen

Secondary School Clinician Characteristics

At the time the data were analyzed, the AT-PBRN consisted of 23 secondary school clinicians (men = 7, women = 16, age = 27.3 ± 7.0 years). Most clinicians (n = 18, 78.3%) had a 4-year college degree and 21.7% (n = 5) had a master's degree as the highest level of education. The majority of clinicians had been certified for less than 2 years (n = 13, 56.5%), with 21.7% (n = 5) being certified for 3–5 years, 13.0% (n = 3) for 6–10 years, and 8.6% (n = 2) for more than 10 years. Most had been employed at their current CPS for less than 2 years (n = 20, 87.0%), whereas 13% had been employed at their CPS for 6–10 years. The title or position of participating clinicians was primarily graduate assistant/resident (n = 12, 52.2%) or head athletic trainer (n = 9, 39.1%); 8.6% (n = 2) were assistant athletic trainers.

Secondary school clinicians were typically employed on a part-time (n = 11, 47.8%) or full-time (n = 7, 30.4%) basis through the school or through a clinic-based outreach program (n = 5, 21.7%). The majority of clinicians reported to an athletic director for their duties as an athletic trainer (n = 15, 65.2%), with 30.4% (n = 7) reporting to the head athletic trainer and 4.3% (n = 1) reporting to the student activities director. Most clinicians indicated they worked 20–30 hours per week (n = 12, 52.2%), with 17.4% (n = 4) working less than 20 hours, 17.4% (n = 4) working 31–40 hours, and 13.0% (n = 3) working more than 40 hours. None of the clinicians had teaching duties at the school. On average, clinicians had been members of the AT-PBRN for 9.3 ± 3.9 months.

Secondary School Patient Characteristics

Between September 2009 and April 2011, 2523 patients were entered into the EMR. Patients had a mean age of 15.9 ± 1.3 years. Males represented a larger percentage of total patients than females (n = 1500 [59.9%] versus n = 1023 [40.5%], respectively). Patients were fairly evenly distributed across all grade levels: freshmen (n = 735, 29.1%), sophomores (n = 741, 29.4%), juniors (n = 563, 22.3%), and seniors (n = 484, 19.2%). The majority of injured patients participated in football (n = 407, 34.2%), followed by soccer (n = 209, 17.6%), basketball (n = 147, 12.4%), track (n = 107, 9.0%), wrestling (n = 61, 5.1%), and baseball (n = 56, 4.7%), with the remaining 18% (n = 214) participating in 10 other sports.

The daily sign-in log records show a total of 3140 patient encounters between December 2010 and April 2011, with the majority relating to care sought for an athlete's current injury (n = 1925, 61.3%). Care sought for preventive services (n = 754, 24.0%) was the second most common reason noted in the daily sign-in, followed by encounters relating to a new injury (n = 461, 14.7%).

Among the patients entered into the system, 877 encounters consisted of a complete initial evaluation, with the remaining patient encounters documented as daily treatment or rehabilitation sessions, follow-up evaluations, or discharge notes. The percentages and frequencies of the most common diagnoses recorded at the time of the initial evaluation are shown in Table 3.

Table 3.  Most Frequent Injury Diagnoses Recorded Across All Patients at Time of Intake (n = 877)

View Fullscreen

Secondary School Practice Characteristics

A total of 1491 treatments or procedures were reported at the initial evaluation and were recorded into the patient-oriented EMR system. The percentages and frequencies of the treatments or procedures provided at the time of the initial evaluation are listed in Table 4. The treatments and procedures provided to athletes with regard to their daily treatment status are described in Table 5, and the percentages and frequencies of treatments for each of the top diagnoses are listed in Table 6. These percentages were calculated independently for each diagnosis, using the number of total treatments as the denominator. The median number of treatments per injury was 3 (interquartile range = 2, 4; range = 2–19).

Table 4. Evaluations and Treatments or Procedures Recorded at Initial Evaluation (n = 1491)

View Fullscreen

Table 5.  Daily Sign-In Log Evaluations, Treatments, and Procedures for New Injuries, Current Injuries, and Preventive Services (n = 3140)

View Fullscreen

Table 6.  Daily Evaluations and Procedures for the Most Prevalent Diagnoses^a

View Fullscreen

Secondary School CPS Characteristics

Currently, no published athletic training data describe a typical secondary school setting. The most inclusive survey of secondary school athletic training practice is the National Sports Safety Secondary School Benchmark (N4SB) study²² of 4232 athletic trainers commissioned by the National Athletic Trainers' Association. In that unpublished report, 80.9% of respondents worked in a public school and 13.2% in a private or parochial school, with 57.7% of schools having more than 1000 students enrolled and 42.3% having a student enrollment of less than 999. Data from the N4SB study suggest that 71.8% of schools had fewer than 399 male athletes, while 26% reported more than 400 male athletes. A total of 79.4% of respondents indicated that their school had fewer than 399 female athletes, and 18.6% had more than 400 female athletes.²² The secondary schools in the AT-PBRN are similar to those in the N4SB sample with respect to the public and private or parochial status and school enrollment but tended to have smaller numbers of both male and female athletes.

Despite similarities to the N4SB sample, the AT-PBRN is limited geographically because the majority of its secondary school CPSs are located in Arizona and the northeast region of the country. In order to achieve its overarching mission of improving quality of care and patient outcomes, the AT-PBRN must establish a collection of geographically diverse and nationally representative secondary school CPSs throughout the country. This is important for future studies of practice characteristics, comparative effectiveness research, and injury surveillance and will allow for comparisons of injuries, treatments, and interventions across geographic location and socioeconomic status.

Secondary School Clinician Characteristics

The secondary school clinicians represented in this study were primarily young professionals who had completed their bachelor's degrees and were currently pursuing master's degrees. Their average age was lower than the mean age of the N4SB respondents (27.3 ± 7.0 years versus 35.7 ± 9.9 years, respectively), and only 21.7% held a master's degree, compared with 55% of the N4SB sample.²² Most secondary school clinicians practicing in the AT-PBRN had less than 5 years of clinical experience (78.2%); only 30.7% of the N4SB sample reported practicing for less than 5 years. These findings are not surprising considering that the AT-PBRN partnered with postprofessional athletic training education programs to enroll secondary schools associated with these educational programs during the initial development of the AT-PBRN. Athletic trainers joining the AT-PBRN during their postprofessional studies provide a potential advantage to future data-collection efforts, as they are being trained and socialized to use the patient-oriented EMR as part of their educational programs and have translated that to use in their clinical practice. One of the aims of the AT-PBRN is to produce consistent use of the EMR with these graduate student clinicians. That is, we hope that, with ongoing education, these clinicians will continue to use the patient-oriented EMR and participate in the network once they earn their master's degrees and find full-time employment.

Clinicians participating in the AT-PBRN also tend to be younger and have practiced fewer years than clinicians in other PBRNs. For example, only 5% of clinicians in the Northwest PRECEDENT Dental PBRN were 30 years of age or younger, and 12.9% of dentists had been in practice for 5 years or less.²⁰ Similarly, only 28% of Dietetics PBRN members reported 7 or fewer years of clinical experience.²¹ Experience has been shown to play a role in the interest of clinicians to participate in a PBRN. One study of nurse practitioners (NPs) found that those NPs who were interested in PBRN participation had 9.3 ± 7.6 years of experience as NPs and 11.0 ± 7.7 years as RNs before becoming NPs.²³ Differences in age and years of experience may be a direct result of the professions involved in the other PBRNs. Both dentistry and NPs have post-baccalaureate entry-level degrees; many athletic trainers take the Board of Certification examination and practice after completing a bachelor's degree, whereas fewer take the entry-level master's route to certification.

Although educational requirements may explain some of the differences in clinician characteristics among the health care professions, data from the N4SB sample suggest that the AT-PBRN may not currently capture a nationally representative sample of athletic trainers and may include a greater proportion of CPSs that are affiliated with postprofessional AT education programs. As mentioned previously, a nationally representative sample of CPSs and clinicians is essential to the long-term goals of the AT-PBRN. In order to address this limitation, the AT-PBRN has expanded its recruiting efforts to include CPSs that are not associated with postprofessional AT education programs. In fact, none of the 22 CPSs added to the AT-PBRN since the end of data analysis are associated with a postprofessional AT education program.

Secondary School Patient Characteristics

Injury-surveillance data in the secondary school setting are currently available, with efforts focused primarily on particular sports,^24–30 types of injuries,^31–34 participation levels (practice versus competition),³⁵ and legality of sport maneuvers.³⁶ However, these studies often do not track the patient after the initial injury to determine the type, frequency, or duration of subsequent treatments. Better identification and reporting of the patient characteristics and types of injuries evaluated and treated by athletic trainers are important for many professional areas, including injury-prevention efforts and policy development. Injury surveillance is the first step in the model to study risk factors and interventions,³⁷ and investigations addressing these issues through surveillance methods are much needed.³⁸ The Reporting Injuries Online (RIO) studies^{25,26,29–31} and others²⁷ have produced invaluable information regarding injury rates and risk factors in interscholastic athletes; however, these data are not linked in a meaningful way to patient care and clinical outcomes. Therefore, although these data are important for understanding injury trends and improving safety in the secondary school setting, the fact that they are collected independently of the health care services provided by athletic trainers severely limits their use.

To our knowledge, the only published investigation of athletic training outcomes is from Albohm and Wilkerson.³⁹ In that investigation, patients were older (27.3 years) than those in the present study (15.9 ± 1.3 years), but similar to the current study, 60% were male.³⁹ Not surprisingly, this sex bias toward males may be explained by the highest number of patient encounters documented within the AT-PBRN being with football athletes. According to the National Federation of State High School Associations 2009–2010 athletics participation survey,¹² football was the boys' sport that had the largest number of participants, with more than 1.1 million student-athletes. Further, studies of secondary school sports injury epidemiology have shown that football players also sustain the highest injury rates.^27,34,40,41

Although we did not aim to address injury rates and risks, variables often found in surveillance and epidemiologic studies, the best comparative data come from epidemiologic studies of secondary school sports injuries. In a 3-year study of high school sports injuries, Powell and Barber-Foss⁴¹ reported 23 566 total injuries (59.9% to the lower extremity and 20.8% to the upper extremity) among athletes in 150 secondary schools. This equates to approximately 52 reportable injuries in each of the participating schools for each year of the study. The data collected within the EMR for the current study resulted in approximately 79 documented injuries per school per year. Discrepancies in the reported injuries between the studies may result from methodologic differences, including the definition of injury, which should be considered when reviewing the data. However, the larger point is that the data-collection system within the AT-PBRN is capable of documenting the large number of injuries evaluated by athletic trainers. We as well as Powell and Barber-Foss⁴¹ reported that football, soccer, and basketball resulted in more reportable injuries than other sports. Another area of interest is body location of injury. Fernandez et al³⁴ found that football for boys and soccer for girls resulted in the most lower extremity injuries over the course of 1 year in a nationally representative sample of 100 secondary schools. In this study, 4350 injuries were reported, with 52.8% of these affecting the lower extremity. The most common diagnoses reported were ligament sprain (95% of all lower extremity injuries), strain (17.1%), contusion (12.1%), fracture (5%), and tendon strain (3.1%).³⁴ These injuries were primarily reported at the ankle (40.3%) and knee (25.3%). Our data are similar in that ankle sprain/strain was the most common injury diagnosis.

Although these epidemiologic investigations have begun to describe the incidence of injuries in different sports, they often do so by categorizing injuries either by body part or injury type,^34,40–42 which makes it difficult to evaluate the occurrence of specific injuries, such as ankle sprains. Including an EMR as part of the data-collection infrastructure allows all injuries to be linked directly to an ICD-9 code to provide more precise information about the specific injuries being evaluated and treated by athletic trainers.

Secondary School Practice Characteristics

Establishing the clinical outcomes and effectiveness of athletic training services is a critical issue facing the profession.^43–48 Lack of information regarding the effectiveness of athletic training practice has been highlighted,^43,46,49 yet obtaining this important information takes time, training, and resources. Agencies such as the National Institutes of Health and Agency for Healthcare Research and Quality recognize these obstacles and have advocated the development and implementation of large-scale PBRNs, which take advantage of strong academic and community partnerships and shared resources and expertise. The PBRN infrastructure provides a highly successful mechanism for determining the clinical outcomes of care provided for numerous conditions and for translating research findings into clinical practice, ultimately improving patient care.^50–52

To date, successful, large-scale efforts to obtain clinical outcomes data regarding athletic training services in any setting have been limited.^39,53 Albohm and Wilkerson³⁹ reported the largest multisite effort to evaluate the effectiveness of athletic trainers for improving HRQOL outcomes of their patients. They evaluated athletic training practice in a variety of health care settings (clinics, secondary school athletic training clinics, college or university athletic training clinics, and industrial settings) across the country. These data provide preliminary evidence that athletic trainers are effective at improving the function and HRQOL of their patients; however, they did not provide additional information related to athletic trainers' practice characteristics beyond stating that 90% of all treatments were performed by athletic trainers.³⁹

In the current investigation, the most common treatments provided were hot or cold pack and strapping or bracing, with less common treatments related to therapeutic exercise, massage, and modalities. These results are not surprising given the typical schedule of a secondary school athletic trainer. Significant social pressure is focused on quickly preparing athletes for practice, and little time is devoted to services that are more time consuming, such as manual therapy. In addition, injured athletes often do not report to practice and may be absent until the injury is healed, providing less opportunity for delivery of treatments such as functional exercise. The availability of athletic training supplies in the secondary school setting may also affect reported treatments. For example, not all schools have access to therapeutic modalities, such as electric stimulation, which may make their use in daily clinical practice less likely and, as a result, less often reported. Data are not available to evaluate the reasons for the practice patterns observed in this study. Future research is warranted to investigate the influences upon secondary school athletic training practice.

Another interesting finding was that of preventive treatments performed by the athletic trainer, captured through the daily sign-in log. Nearly half of all athletes who signed in for injury prevention received athletic training treatment, and almost 65% of individuals received taping or strapping. We are unaware of any other investigation that has captured the types of preventive treatments athletic trainers provide to secondary school athletes. These data will be helpful in characterizing the components of athlete care that often go unreported in the medical record because the athlete is uninjured.

Of particular interest regarding treatment data, although there were 877 new injuries recorded in the EMR, only 53.9% of these were coded for receiving an athletic trainer evaluation. All new injuries should have received a code for evaluation by an athletic trainer. Interestingly, among diagnoses and associated treatments (Table 6), concussive injuries were the most frequent (87.8%) injury to receive an injury evaluation by the athletic trainer code. In contrast, the most commonly diagnosed injury, the ankle sprain or strain, received the athletic trainer evaluation code for fewer than half of the reported injuries. Lack of coding for athletic trainer evaluation was also noted in the daily sign-in log, with less than half of new injuries receiving the evaluation code. The reason for this inconsistency warrants further investigation, but the coding of athletic training services is rarely required by employers because records are not being scrutinized by third-party payers, which may be part of the reason for poorly coded records. However, the finding that concussions were more often coded correctly with the athletic trainer evaluation code suggests that documenting and coding a thorough evaluation for some injuries may be viewed as more meaningful or important than with other, perhaps less serious conditions (eg, sprains).

In addition to the poor coding for the actual evaluation by the athletic trainer, poor coding was noted for other treatments as well. For example, approximately 19% of patients with sprains and strains reportedly received an ice or hot pack. This percentage is likely low given the common practice of rest, ice, compression, and elevation for the treatment of acute injuries, especially ankle sprains and strains. Clinicians may view treatments such as ice to be standard components of care that do not warrant detailed reporting, although this idea warrants further study. This is one area in which the AT-PBRN must further emphasize in training and education provided to participating clinicians that all treatments be appropriately documented within the EMR and that clinicians understand the value and benefit of complete documentation.

Concerns about medical coding are not new, even in professions with a long history of coding practice.⁵⁴ One study⁵⁵ reported on the accuracy of administrative data in health records, focusing specifically on diagnosis information, and found that the primary diagnosis was correctly recorded in only 57% of visits. Errors in coding were attributed to physician diagnostic error, missing forms, and inaccurate data entry. Lack of coding for standard, routine components of athlete health care, such as the actual injury evaluation or the administration of ice for an acute injury, is not surprising. The majority of secondary school athletic trainers do not bill for services; thus, they have less incentive to code their care and may not fully appreciate the value coding could bring to their practice. Incomplete documentation is problematic, and efforts should be made to better educate and inform athletic trainers as to the importance of documentation and proper medical record coding to facilitate accuracy in these essential health records.