Methods of the National Athletic Treatment, Injury and Outcomes Network Surveillance Program (NATION-SP), 2014–2015 Through 2018–2019
The National Athletic Treatment, Injury and Outcomes Network Surveillance Program (NATION-SP) was established in 2011 to provide a comprehensive appraisal of injuries sustained by high school student-athletes who received services from athletic trainers (ATs). The purpose of this article is to update the surveillance methods of the NATION-SP for data reported during the 2014–2015 through 2018–2019 academic years. The NATION-SP used a rolling recruitment model to identify a convenience sample of US high schools with access to ATs. The ATs at participating institutions volunteered to contribute data via electronic medical records systems; common data elements were then pushed to and maintained by the Datalys Center for Sports Injury Research and Prevention. The ATs completed detailed reports on each injury, including the condition and circumstances. The treatments component was used to comprehensively assess the services provided to athletes by ATs. The outcomes companion component was developed to monitor patient-reported outcomes after athletic injury. The NATION-SP continues to serve a critical purpose in informing injury-prevention and treatment efforts among high school athletes.Background
Surveillance System Structure
Summary
Sports injury surveillance plays an important role in injury prevention, particularly in high school (HS) sports, given the large volume of HS student-athletes in the United States.1–3 Over the past 2 decades, surveillance systems such as High School Reporting Information Online and the National Athletic Treatment, Injury and Outcomes Network Surveillance Program (NATION-SP) have collected broad information on sport-related injury incidences and outcomes in this population.4,5 Established in 2011, the NATION-SP is a surveillance program developed to provide a comprehensive appraisal of injuries and illnesses sustained by HS student-athletes who received services from athletic trainers (ATs). Since its inception, the NATION-SP has captured both time-loss (TL) and non–time-loss (NTL) injuries in HS sports. According to these data, the majority (approximately 83%) of all injuries reported in HS student-athletes were NTL injuries.5 Furthermore, the NATION-SP has also afforded insight into the various treatments administered by ATs practicing in the secondary school setting, as well as patient-reported outcomes after HS sport-related injuries.6,7 The distinctive features of the NATION-SP uniquely position the program to support the process of injury prevention among HS student-athletes.
Surveillance methods continue to evolve with time. Technological advances coupled with progress in sports medicine research have resulted in novel methods for conducting surveillance and enabling new accommodations in data collection. This evolution has been previously characterized in other surveillance systems, such as the National Collegiate Athletic Association Injury Surveillance Program.8,9 Given that the NATION-SP is approaching its tenth year of operation, it is important to note the most up-to-date methods of this surveillance system. The purpose of our article is to characterize the operational methods of the NATION-SP for the 2014–2015 through 2018–2019 academic years, which vary from previously published methods.
SURVEILLANCE METHODS
The NATION-SP is funded by the National Athletic Trainers' Association (NATA) and maintained by the Datalys Center for Sports Injury Research and Prevention. Although the general methods are comparable with those of the National Collegiate Athletic Association Injury Surveillance Program, the NATION-SP also includes components investigating the treatments administered by ATs and quantifying patient-reported health outcomes for a subset of reported injuries.6–9 Together, these features capture a multidimensional perspective on the epidemiology of sport-related injuries and sports medicine operations in the HS setting. The NATION-SP was reviewed and approved by the Western Institutional Review Board (Puyallup, WA).
Data Collection
The ATs at participating institutions contributed data by entering information into electronic medical record (EMR) systems or injury-documentation applications. The NATION-SP uses the common data element (CDE) strategy, which allows data to be exported from 3 EMR systems and injury-documentation applications: the Athletic Trainer System (Keffer Development Services), the Sports Injury Monitoring System (FlanTech Computer Services), and the Injury Surveillance Tool (Datalys Center). The CDE export standard allows ATs to document injuries as part of their normal daily clinical practice instead of duplicating efforts to report injuries for the sole purpose of participating in an injury-surveillance program. The ATs completed detailed reports on each injury, including the condition (eg, site, diagnosis) and circumstances (eg, activity, mechanism, event type [eg, competition, practice], playing surface). They were able to view and update previously submitted information during the course of an academic year. The ATs were not financially compensated for their data-collection efforts; however, they could claim continuing education units for their participation during each reporting period.
The EMR software vendors participating in the NATION-SP underwent a certification process involving modifying their systems and embedding secure data-transmission protocols to allow the transfer of deidentified records to secure Datalys Center servers. Before export, the data were stripped of identifying information, tagged with a unique 16-digit alphanumeric code, and encrypted. This process is Health Insurance Portability and Accountability Act (HIPAA) compliant. All incoming data were evaluated through a quality control (QC) process, and Datalys Center staff assisted ATs in resolving any concerns regarding invalid values.
Participating Schools
The NATION-SP used a rolling recruitment model to identify a convenience sample of high schools with access to ATs, and participation varied between 2014–2015 and 2018–2019 (Table 1). The ATs at participating institutions selected specific sports for which they contributed exposure, injury, and treatment information to the surveillance system. A list of sports included in the NATION-SP between 2014–2015 and 2018–2019 is provided in Table 2. The ATs were recruited to participate using multiple strategies, including in-person recruitment at the NATA Annual Meeting and Clinical Symposia and communication via the Datalys Center website, emails, social media, and word of mouth. The certified software vendors also supported our recruitment efforts.
Definitions
Number of Athletes
The number of athletes participating in an organized HS-sanctioned athletic event for a school-sponsored sport was defined as the number of athletes present at the activity who were at risk for injury due to their participation. A nonparticipant was defined as any athlete not participating in the team activity due to an injury or some other conflict (eg, class, physician appointment).
Sport-Related Injury
A reportable injury was defined as an injury that (1) occurred as a result of participation in an organized HS-sanctioned athletic event for a school-sponsored sport and (2) required attention from an AT or physician, regardless of time loss. The ATs were able to report multiple injuries that occurred from 1 injury event. A time-loss (TL) injury was any injury evaluated or treated by an AT or physician for which an athlete returned the day after the date of injury or beyond. A non–time-loss (NTL) injury was any injury evaluated or treated by an AT or physician for which an athlete returned to participation on the date of injury.
Quality Control
The process of verification and review of incoming data was an essential component of the NATION-SP. Before data were entered in the research database, they were checked for accuracy and completeness via a process automated by a proprietary verification engine (VE). The VE identified partial or failed submissions due to errant or missing values. Datalys Center staff regularly conducted an additional series of inspections and subsequently contacted the ATs for assistance in reconciling any data quality concerns. All data reported to the NATION-SP were considered final as of June 30 of each academic year. Any records submitted or modified after that date were not included or reflected in the research datasets.
Data Management
Exposures
Exposure data were considered valid if (1) they occurred between July 1 and June 30 (during the academic year) and (2) the number of athletes participating in the reported event was not zero or missing. Zero or missing values for the number of athletes remaining after the QC process were replaced with mean imputations. These mean values were estimated from all valid exposure data from the same year, school, sport, and exposure event type. If every value for the number of athletes was missing for the year, school, sport, and exposure event type, the mean values were estimated from all reporting schools for that year, sport, and event type. In comparison with previous years of the NATION-SP, during 2014–2015 through 2018–2019, competition schedules posted on school or association Web sites were used to confirm accurate reporting of the season variable (ie, preseason, regular season, postseason, or out of season); the season and event type were updated if necessary.
Injury Data
Injury events with multiple reported injuries were identified and evaluated for duplicate submissions. These injuries were retained if each had a separate specific injury diagnosis or a different affected body part (eg, ankle and knee injuries occurring in the same injury event). Otherwise, duplicate injuries were removed. The VE contained a validation process to ensure agreement between a categorical injury outcome variable that measures days missed due to an injury and the date of return to participation; ATs were asked to assist in reconciling any discrepancy during the QC process. As an addition to the existing practices from 2014–2015 through 2018–2019, if a discrepancy remained after the VE check and QC process, both the injury outcome and the date of return were set to missing as there was no valid method of reconciling the inconsistency.
Qualifying Criteria for Inclusion
The qualification criteria used for 2011–2012 through 2013–2014 were modified during 2014–2015 through 2018–2019. At the completion of the academic years from 2014–2015 through 2018–2019, data from each team were reviewed to determine whether they qualified for inclusion in the reportable dataset. To qualify, at least 1 injury must have been reported for the team during the athletic season unless otherwise confirmed by the AT (ie, zero injuries in a season). Due to variability among HS practice and competition schedules, the number of submitted exposures was not considered when determining qualification.
Treatments Component
To comprehensively record the services provided to athletes, the ATs captured the number of unique days on which an athlete received 1 or more treatments in an academic year and a summary of AT services provided for reported injuries, nonathletic injuries (or problems), and preventive treatments in the Injury Surveillance Tool. An AT service was defined as the application of any type of manual therapy, modality, exercise and evaluation, testing, or skills session that the player received by interacting with the AT. A complete list of AT services captured as part of the treatments component is provided in Table 3. It should be noted that the corresponding Current Procedural Terminology codes associated with these services are not listed in the NATION-SP. These data were streamed to a database separate from the exposure and injury records and did not undergo QC. Only treatments associated with injuries retained in the research datasets were included in the research treatment datasets; however, all reported nonathletic and preventive treatments were included.
Outcomes Companion Component
The outcomes component augments routine injury surveillance in the NATION-SP. An online investigator-initiated survey application external to the system QC and data-management processes was specifically developed to enable this feature. The application was designed to be triggered when an injury meeting preestablished criteria was reported. The internal trigger prompted the AT-reporter to secure responses to additional patient-reported outcome measures from the injured athlete at predetermined intervals.
SUMMARY
The NATION-SP serves a critical purpose in informing injury-prevention efforts among HS athletes. Due to its ability to capture both TL and NTL injuries, the NATION-SP most closely represents the complete spectrum of sport-related injuries occurring in this population. Further, the distinct features of the treatment and outcomes components afford unique insight into injury-management practices and injury outcomes in HS sports. These components provide not only a momentary snapshot of current practices in the athletic training facility but also a platform on which the evolution of AT practices and services can be better appraised. The operational characteristics of the NATION-SP ensure timely data input with minimal burden to the reporting clinician. The postsubmission QC and data-management processes ensure that the data retained in the research datasets are of the highest possible fidelity. It is important to note that the updated surveillance methods described here are not a departure from the initial global concept of the NATION-SP; rather, these updates were deployed in the interest of improving data reporting and, ultimately, retention in the analysis datasets. These practices facilitate the provision of a stable foundation from which to identify emerging injury-related patterns and develop injury-prevention strategies. However, the NATION-SP is not without limitations. Most notably, these data are not generalizable to the national population of HS athletes, not only because sampling weights are not estimated as part of the study design but also because not all high school athletics programs have access to ATs. In addition, given the definition of reportable injury noted herein, reporting elements are left to the discretion of the ATs for inclusion. This could lead to potential heterogeneity in the reported observations, particularly regarding illnesses and infections.
Although the current practices are aimed at ensuring that valid data are collected as close to real time as possible, existing methods continue to present a nonnegligible burden to clinician-reporters, particularly with regard to capturing exposure-related CDEs that may not represent a meaningful component of routine clinical practice. Exposure ascertainment in sports injury surveillance continues to remain an area warranting further methodologic advancement. In addition, it is necessary to adapt surveillance methods over time, targeting specific areas for improvement and augmenting methods to fully capitalize on technological advances. Whereas extracting CDEs from existing commercial software is an effective data-reporting framework, the success of this framework inherently depends on the assumption that QC and data management evolve in alignment with technological adaptations in commercial injury-tracking software. As systems and software evolve, the process of data integration and management needs to align with those adaptations. It is important to acknowledge that these processes are innately nonstationary, and the processes of data qualification, quality control, and data management have evolved markedly since the inception of the NATION-SP. Therefore, further adaptations in this regard are to be expected over time in order to facilitate updates and augmentation. Ultimately, such a continuous process will ensure that the NATION-SP and comparable sports injury-surveillance systems continue to support the health and safety initiatives that are targeted at this population.
Contributor Notes
Authors Sarah N. Morris and Avinash Chandran contributed equally to manuscript preparation.