Going Beyond the State Law: Investigating High School Sport-Related Concussion Protocols

Study Participants and Data Collection

We used a qualitative document analysis study design to investigate the components of SRC protocols at Pennsylvania high schools during the 2018–2019 academic year. This work was categorized as “not human subject research” by Duquesne University, and thus, institutional review was not warranted. The Pennsylvania Interscholastic Athletic Association (PIAA) provided us with a current list of all high schools participating in sports under their jurisdiction. We invited all PIAA high schools (n = 757) to participate in this study through an informative email sent to the superintendents, high school principals, and athletic directors. Approximately 3 weeks later, a round of follow-up emails was sent to high schools that did not respond to our initial communication. Participation was voluntary and indicated when an electronic copy of the high school's SRC protocol was provided to the research team. We also asked each high school's responding representative if the athletics program had an AT (yes or no), and if yes, to indicate the level of sports medicine coverage provided (full time or part time). From July 2018 to January 2019, we had a response rate of 53.4% (n = 404/757). Of those, 17.6% (n = 71) declined participation, 6.4% (n = 26) replied that they did not have an SRC protocol, 12.1% (n = 49) responded but did not include any usable information, 18.3% (n = 74) indicated that they had an SRC protocol but did not provide a copy of it to the research team, and 45.5% (n = 184) sent an electronic copy of their SRC protocol. The 184 SRC protocols assessed in this study accounted for 24.3% of all Pennsylvania high schools.

Protocol Assessment Instrument

We developed a component analysis guide to complete the artifact analyses of the included SRC protocols. The component analysis guide was informed by the Pennsylvania Safety in Youth Sports Act,¹⁵ the 2017 Concussion in Sport Group international consensus statement on concussion in sport,¹ the National Athletic Trainers' Association position statement on the management of sport concussion,² the NCAA interassociation consensus document on diagnosis and management best practices,¹⁶ and the NCAA “Concussion Safety Protocol Template.”⁹ The items included in the component analysis guide were also consistent with content found in additional SRC management sources, such as the National Federation of State High School Associations' suggested guidelines for management of concussion in sport¹⁷ and the American Medical Society for Sports Medicine's position statement on concussion in sport.¹⁶ The initial component analysis guide was composed by the principal investigator (E.B.) and reviewed and revised by 2 additional members of the research team (ie, 1 concussion expert [T.C.V.M.], 1 qualitative research expert [C.E.W.B.]). The updated version was then circulated to 4 external individuals to assess face validity and comprehension. The team of external reviewers consisted of 1 concussion researcher with a career in academia (T.C. [see Acknowledgments]), 1 clinical specialist who oversees concussion management for the athletics department and general health services at a university (B.V.), and 2 licensed ATs who practice in Pennsylvania (K.B., G.J.). We considered the feedback provided by these qualified individuals and completed a second round of revisions on the assessment.

The final component analysis guide contained 67 items. The first 7 items specifically assessed compliance with the Pennsylvania Safety in Youth Sports Act.¹⁵ The Safety in Youth Sports Act¹⁵ is Pennsylvania's SRC state law that was signed into law in 2011 and provides 5 mandatory tenets. The act requires high schools to make various SRC educational documents available to student-athletes and their parents or legal guardians. In return, these individuals must acknowledge receipt of the information, stating they reviewed the material and understand the risks of sport participation. The most important components of the law are removal of play for any athlete showing the signs and symptoms of an SRC and no return to athletic participation until written clearance is supplied by an appropriate medical professional. The state law defines an appropriate medical professional as “a licensed physician who is trained in the evaluation and management of concussions or a licensed or certified health care professional trained in the evaluation and management of concussions and designated by such licensed physician.”¹⁵ The act also requires coaches to complete SRC training courses before each sport season and outlines that school districts must develop and implement penalties for coaches who do not abide by the removal and RTP state law tenets. A mandatory state law component score (maximum = 5) was determined based on the number of mandatory items included in each protocol.

The remaining 60 items were used to determine a total component score of the number of included items in the following areas: education, preparticipation assessment, prevention and reducing exposure to head trauma, on-field recognition, off-field assessment, general follow-up care, RTL, and RTP. The maximum total component score was 60, and higher scores indicated a greater number of included protocol items. These items were further broken down into subscores of preparticipation and prevention components (maximum score = 20), recognition and assessment components (maximum score = 19), and management components (maximum score = 21).

Artifact Analysis Procedure

Four ATs (N.H., C.P., R.G., I.S. [see Acknowledgments]) independently analyzed the content of the 184 high school SRC protocols. At the time of protocol analysis, all assessors were practicing in the high school setting, and their clinical experience ranged from 1.5 to 5 years. These individuals were not involved in the development of the assessment instrument. Before analyzing the documents, the assessors completed a 1-hour initial training session with the principal investigator. This training consisted of an item-by-item breakdown of the component analysis guide that included an explanation of the intended interpretation for each item. After the initial training session, all assessors independently completed a practice evaluation of the same protocol (protocol 1). Feedback was provided to the assessors regarding 26 items that had less than 100% agreement (7 items with 50% agreement; 19 items with 75% agreement). Two additional protocols were scored by all assessors and used to determine interrater reliability. The Fleiss κ values indicated that protocol analysis item agreement among the 4 assessors was good (protocol 2: κ = 0.75 [95% CI = 0.66, 0.84]; protocol 3: κ = 0.74 [95% CI = 0.65, 0.83]).

For document analysis, each protocol was given a unique number code and then randomly allocated to an assessor. Each protocol was evaluated by 1 assessor, and each assessor performed artifact analyses of approximately 35 to 50 SRC protocols over a 60-day period. For each protocol, all outcome data were recorded on a deidentified component analysis guide and submitted for data entry. The outcome data from each protocol were then entered into a spreadsheet by a research assistant. In addition to the artifact analysis outcomes, the sports medicine coverage information provided by the responding high school representative and the school demographic information obtained through the PIAA (ie, PIAA district [1–12], school type [private, public], total number of enrolled students) were also recorded in the spreadsheet.

Data Analysis

Descriptive statistics were computed for the demographic information of participating high schools. Frequencies were used to describe the proportion of included SRC protocols for each item assessed in the component analysis guide. Means and standard deviations for the entire sample were calculated for the mandatory state law component score, total component score, preparticipation and prevention subscore, recognition and assessment subscore, and management subscore. The mean total component score was also broken down by PIAA district. Additionally, Mann-Whitney U tests were conducted to investigate differences in scores (median [interquartile range]) for full-time versus part-time sports medicine coverage for the mandatory state law component score, total component score, preparticipation and prevention subscore, recognition and assessment subscore, and management subscore. Although kurtosis and skewness values for the component score variables were within normal limits, the data did not appear to be normally distributed based on histograms constructed for each component score by sports medicine coverage group. Therefore, nonparametric statistical approaches were employed. Statistical significance was set a priori at P < .05. All analyses were completed using SPSS (version 26; IBM Corp).

Sample Demographics

Of the 184 high schools that submitted protocols for inclusion in this study, 87.5% (n = 161) were public, and 12.5% (n = 23) were private. The sample had a median (interquartile range) student population of 526 (690). The PIAA districts with the most included schools were 3 (20.7%, n = 38), 1 (16.8%, n = 31), and 7 (13.0%, n = 24). The PIAA districts 8 (1.1%, n = 2), 2 (2.7%, n = 5), and 9 (2.7%, n = 5) were the least represented (Figure). The vast majority of included high schools reported that they had an AT (99.4%, n = 178/179 responses), and of those, 92.0% (n = 149/162; missing, n = 16) indicated they had full-time sports medicine coverage. High school demographic information by study response category can be found in Table 1. Although the 26 high schools that indicated they did not have an SRC protocol were not included in any analyses, it is pertinent to note that 19.2% (n = 5) of those schools did not have an AT, 34.6% (n = 9) reported having a part-time AT, and 38.5% (n = 10) were in PIAA districts 5 and 6.

View Figure

• Download as Powerpoint
• Download Figure

Table 1 High School Characteristics by Study Response Category

View Fullscreen

The SRC Protocol Components

Only 41.3% (n = 76) of SRC protocols directly mentioned or referenced the Pennsylvania state law (Table 2). Moreover, 23.4% (n = 43) were considered fully compliant with the state law. The most included state law item was written medical clearance before RTP (71.7%, n = 132), and the least common inclusion was penalties for coaches in violation of the removal and RTP guidelines (25.0%, n = 46). The mean number of mandatory state law items included in the protocol sample was 2.5 ± 1.8 (50.6% of 5 possible). Additionally, an average of 15.6 ± 9.8 (26.0% of 60 possible) total components were present in the SRC protocol sample. The Figure provides the total mandatory state law component score and total component score by PIAA district. The subscore results were as follows: 4.2 ± 3.6 (21.0% of 20 possible) preparticipation and prevention components, 5.0 ± 4.1 (26.3% of 19 possible) recognition and assessment components, and 6.4 ± 4.3 (30.5% of 21 possible) management components. See Tables 2–5 for a complete frequency itemization of all individual components.

Table 2 Frequency of Safety in Youth Sports Act Components in Pennsylvania High School Sport-Related Concussion Protocols, 2018–2019

View Fullscreen

Table 3 Frequency of Preparticipation and Prevention Components in Pennsylvania High School Sport-Related Concussion Protocols, 2018–2019

View Fullscreen

Table 4 Frequency of Recognition and Assessment Components in Pennsylvania High School Sport-Related Concussion Protocols, 2018–2019

View Fullscreen

Table 5 Frequency of Follow-Up Management Components in Pennsylvania High School Sport-Related Concussion Protocols, 2018–2019

View Fullscreen

The SRC Protocol Components by Sports Medicine Coverage

A difference existed in the median number of mandatory state law components in SRC protocols between high schools that had full-time (3 [1–5]) versus part-time ATs (0 [0–4.5]; 5 possible; U = 647.5, P = .044). When we considered the broader components, schools with full-time sports medicine coverage had more preparticipation and prevention (full time = 5 [1–7] versus part time = 0 [0–4.5]; 20 possible; U = 590.0, P = .018), recognition and assessment (full time = 5 [2–8] versus part time = 0 [0–3]; 19 possible; U = 416.0, P < .001), follow-up management (full time = 6 [3–10] versus part time = 3 [2–4]; 21 possible; U = 493.0, P = .003), and total components (full time = 15 [8.5–22.5] versus part time = 6 [3–10.5]; 60 possible; U = 377.5, P < .001) than those with part-time services.

The Pennsylvania Safety in Youth Sports Act

Regarding the mandatory state law components, most high school SRC protocols highlighted the importance of immediate removal from play and the need for written medical clearance before RTP. Additionally, Pennsylvania law requires SRC education for student-athletes, parents, and coaches, but this element was lacking in more than 50% of protocols. The state law directs governing bodies to establish penalties for coaches who violate the state law and is a key aspect of enforcement and coach accountability, yet only 25% of SRC protocols included language regarding violation repercussions. In total, approximately one-fourth of protocols included all mandatory components of the state law. Therefore, local-level policies may not be written to the rigor necessary to ensure athlete safety as state laws intended. In a similar study, researchers^5,21 investigated protocols from 71 nationally representative high schools and reported that a high percentage of SRC protocols complied with the immediate removal from play, RTP medical clearance, and SRC education tenets of state laws. The proportions of inclusion of the 3 main state law components from the SRC protocols examined in that study were approximately 20% to 35% higher than those observed in our Pennsylvania-specific sample. This significant variation in findings highlights the importance of considering SRC law implementation from both the state and national perspectives. Information gleaned from SRC law implementation from a national sample can help inform a universal minimum standard of care, whereas a more local assessment enables state lawmakers, interscholastic athletic associations, and school districts to assess their unique strengths and needs to develop effective SRC awareness initiatives and increase the quality of injury management in their geographic region based on the resources available.

It is pertinent to note that the introduction of SRC state laws in the United States has had a positive effect on managing this public health concern. From 2008 to 2012, health care utilization for concussion-related injuries in children in states that had legislation in place increased by 92%.²² In Ohio, patients with a concussion sought care approximately 5 days sooner and recovered 14 days quicker on average after the adoption of a pediatric SRC law in 2013.²³ Additionally, LaRoche et al²⁴ found that high school and collegiate athletes in Wisconsin were significantly more likely to report SRCs after the introduction of the state's SRC law, and more than half of athletes in the sample directly cited the state law as a motivating factor for SRC disclosure. In an examination of SRC trends from 2005 to 2016, the authors²⁵ noted that the introduction of SRC state laws and the associated increased publicity may have also increased reporting of new and recurrent SRCs before and for an extended period after law adoption. These outcomes clearly show the effect of these laws, so although the novelty of SRC state laws may have faded over time, the importance of their purpose has not. Highlighting the key components of these legislations in SRC protocols and awareness initiatives should remain a focal point in continuing to promote immediate injury identification and safe RTP processes.

Preparticipation and Prevention Considerations for SRC

Given the cumulative nature and potential short- and long-term concerns associated with concussion, documentation of previous head injuries during preparticipation examinations is imperative.¹ Despite this, fewer than 13% of the SRC protocols outlined the documentation of previous SRC and concussion-related health conditions (eg, migraines, mental health disorders). Beyond the medical history, baseline neurocognitive and symptom assessments were present in approximately 38% to 50% of included protocols, with balance and oculomotor examinations being far less common. Ultimately, a preparticipation SRC assessment may aid in identifying athletes who need further SRC care, education, individualized behavioral sport adaptation interventions, or all of these.

We noted that a little more than 40% of SRC protocols included annual education. The most cited modes of SRC education were information sheets, in-person lectures, or both. Furthermore, 70% of SRC protocols did not require signed acknowledgment of SRC education. Given that an acknowledgment of receipt is required by the state law, this is 1 component that may warrant more immediate attention. Although a handout or online course may be cost effective and fulfill state law and athletic association requirements, these options may not be as effective as in-person training sessions for coaches and parents of athletes.²⁶ Educational strategies that go beyond passive information, including face-to-face interactive lectures, peer-to-peer instruction, and active learning games, should be considered.^27–29

An increased understanding of SRC consequences may lead athletes to adopt safer styles of play, but no data have supported education alone as a primary prevention strategy for SRCs.⁴ Due to the lack of evidence-based SRC prevention interventions available at this time, it is not surprising that only 3 protocols in our study addressed this facet of SRC care. As high schools seek to add prevention strategies to their SRC protocols, a number of factors should be considered, including the possibility that a multifaceted approach may be more beneficial than focusing solely on protective equipment.³⁰ Evidence is growing in support of behavioral tackling or blocking interventions for reducing head-impact frequency in football players,^31,32 as well as limitations on full-contact practices.^33,34 Another recent prevention strategy is the restriction or elimination of or penalties (or all of these) for sporting maneuvers that carry a high risk of concussion, such as heading in youth soccer³⁵ and checking in peewee ice hockey.³⁶ Given the complexity of this topic and lack of available evidence, high school administrators should partner with their AT, local sports medicine health care providers, or both to determine which prevention strategies are most appropriate to include in their protocols for their athlete population.

Recognition and Assessment of SRC

The use of a multimodal assessment for SRC recognition was outlined in fewer than half of the submitted SRC protocols. Although not part of the state law requirements, this is an area of concern, as multiple assessments focused on different areas of possible dysfunction may more accurately identify individuals with an SRC.¹ Nonetheless, although these results indicate areas for improvement in protocol design, they do not speak to the actions being performed in real-life injury scenarios. For example, even though a written SRC protocol is established, it may not accurately reflect the level of implementation occurring in the system.²¹ Additional SRC recognition and assessment strategies may be used that are not directly outlined in a written protocol. Overall, these findings highlight the need for annual analysis and consistent improvement of SRC protocols to ensure that they reflect current practice recommendations.

Follow-up Management of SRC

With respect to the follow-up management areas of the reviewed protocols, we noted better compliance with these elements. Nearly three-fourths of protocols included a graded RTP progression. These findings are consistent with those reported in New York State, where 78% of schools in Westchester County and 90% of big city school districts contained SRC management procedures in their written policies.⁸ Similarly, the authors of 2 additional studies found that 98.6% of reviewed policies included the RTP tenet⁵ and between 74.6% and 98.6% of schools had specific items that were considered equal to or stronger than the language used in their respective state law.²¹ These higher compliance rates with the RTP items are not surprising because the focus of many state laws has been to avoid premature RTP. However, our analysis identified several areas of less compliance, primarily related to follow-up vestibular or oculomotor assessments and RTL procedures. Fewer than 10% of reviewed protocols mentioned oculomotor or vestibular assessments, compared with more than 50% that addressed neurocognitive assessments. The emphasis on neurocognitive assessments in Pennsylvania is to be expected as one of the most used computerized neurocognitive test platforms was developed in the state. Interestingly, as the emphasis on neurocognitive assessment has decreased in recent years,^1,37 the evidence regarding oculomotor and vestibular deficits has increased.^38–40 It is likely that this newer evidence has not yet been translated into routine clinical practice, nor made its way into current written SRC protocols.

This may also be the case with respect to returning athletes to the classroom, as only one-third of protocols included a statement that academic adjustments may be required. This is contrary to suggestions that some level of academic support should be applied in all cases of concussion⁴¹ and surprising as Pennsylvania is one of the few states with a state-wide program, BrainSTEPS, to assist schools in returning students with acquired brain injuries to the classroom.⁴² As more evidence emerges, it becomes clearer that academic support should be a keystone element of SRC management for student-athletes. Recently, Holmes et al⁴³ found that approximately 84.5% of collegiate athletes and 68.6% of high school athletes reported difficulty concentrating in an academic setting after sustaining an SRC. Furthermore, greater than 50% of both collegiate and high school athletes reported headaches, feeling slowed down, and increased sensitivity to light in the classroom and cited difficulties with math, reading, writing, paying attention, and engaging with digital interfaces.⁴³ The vast majority of high school athletes will not go on to pursue professional sports careers; therefore, returning to academics should be the highest priority at the high school level. Not all athletes will encounter academic challenges after an SRC, but the establishment of a school-level RTL approach could help to identify those who need support and help alleviate the potential negative repercussions that can accompany premature RTL. Due to the lack of RTL approaches in the SRC protocols, we recommend that high school ATs collaborate with school psychologists, counselors, nurses, and physicians to develop a plan that meets the academic needs of the student population they serve.

Sports Medicine Coverage

A risk-reduction strategy to mitigate the mismanagement of injuries and potentially catastrophic events related to sport participation is to employ a full-time AT to provide onsite sports medicine services during practices and competitions. Athletic trainers play a vital role in SRC management, as they are often present from the time of injury through the RTP process. Athletic trainers are also typically tasked with the organization and review of preparticipation examination information for athletes as well as coordinating and documenting SRC education efforts. We were pleased that around 81% of our sample reported that they provided full-time athletic training services to their student-athletes, which was greater than the 67% state-wide total outlined in the Athletic Training Locations and Services Project for the 2018–2019 academic year (https://ksi.uconn.edu/wp-content/uploads/sites/1222/2018/09/ATLAS-2018-Report-Final.pdf). Sport-related concussion protocols from high schools that reported full-time sports medicine coverage contained more mandatory state law tenets and greater than 2 times the total number of components versus schools that employed part-time ATs. Additionally, high schools with full-time ATs included significantly more items for all the subcategories of SRC protocols we assessed. This is encouraging, given that previous researchers⁴⁴ suggested that schools with stronger and more comprehensive wellness policies had greater success with implementation. Although our findings do not represent the actual standard of SRC care provided at the participating high schools, well-developed policies and procedures offer a solid foundation for guiding appropriate evidence-based clinical practices. When ATs are present daily, they can integrate into the school community, survey the unique needs of their athletic population, seek out the resources available to them, identify barriers that inhibit health care implementation and compliance, and ultimately aid in the development of high-quality policies and procedures geared at preserving the health and wellbeing of their patients.

When considering those responding high schools that indicated they did not have an established SRC protocol, approximately 50% reported that they also did not have an AT or only had part-time coverage. These findings highlight the critical role that ATs may play in the establishment, development, and implementation of sports medicine policies and procedures in high schools. This relationship was observed in a study⁴⁵ of Oregon high schools: having an AT was significantly associated with greater implementation of sport-related emergency preparedness recommendations versus schools without an AT. Also, the positive influence of frequent access to an AT was documented in relation to SRC identification and management, as investigators⁴⁶ determined that schools with high AT availability had a greater incidence of reported SRCs, quicker initial assessment times, longer recovery periods, and a higher likelihood of using an RTP protocol than those with low AT availability. These results suggest that having full-time access to an AT may lead to better SRC recognition, quicker care initiation, and more rigorous management approaches.

Protocol Implementation and Barriers

Having a written policy is an important element of compliance with state law and best-practice recommendations, yet simply having an SRC protocol does not equate to implementation of the elements in that document. Differentiating between having an established protocol and implementing that protocol in clinical practice is an important consideration, with the latter being the more important piece in ensuring athlete safety and providing quality health care. Our primary goal was to evaluate the presence of elements of the Pennsylvania state law and SRC awareness and management practice recommendations, but this method did not allow us to identify whether these policies were being implemented or if contributing high schools were using practices that were not outlined in their written policies. Although several studies of SRC protocol compliance exist,^5,8,21 literature regarding implementation in areas of sports safety, including SRC, is limited.^21,47 Sullivan et al²¹ assessed the degree of compliance with state laws and the relationship between policy compliance and implementation. Most written school-level SRC protocols complied with state law components, but self-reported implementation of policy items as well or very well was not ideal, with only 46.2% of schools ensuring that the health care provider granting RTP clearance was trained in SRC management.²¹ Furthermore, in a qualitative study, Davies et al⁴⁷ cited immediate evaluation, notification of key individuals, communication among personnel, and reliance on an AT's assessment as important strategies used to implement SRC laws in the high school setting.

To improve policies, the barriers that inhibit policy implementation should be considered. High school officials (ie, athletic directors, ATs) indicated that tenets of the SRC state laws that required greater resources or were outside the control of the school district, such as the health care provider whom athletes visited for clearance, were less likely to be implemented.²¹ This finding is consistent with evaluations of other safety policies, such as emergency action plans: ATs and athletic directors noted financial barriers to successful policy implementation.⁴⁸ As the most common barriers that inhibit the implementation of the 3 primary SRC state law tenets (ie, education, removal from play, safe RTP) from a sample of 64 high schools from across the United States, Coxe et al⁴⁹ described a lack of quality SRC education, lack of “buy-in” to complete educational requirements, lack of time for educational meetings, lack of communication, resistance from parents and coaches, “old school” sport culture mentality, cost of and access to health care, and a general lack of SRC understanding. Moving forward, school personnel should work together to identify and overcome site-specific barriers to improve sports medicine policy implementation. This collaborative effort should include the creation of an SRC management team, which could include the AT, physicians, athletic director, school nurse, school psychologist, school administrators, etc.

For policies and procedures to affect patient safety and injury recovery outcomes, they must be enforced. Implementation is the act of putting a plan into action, and enforcement is needed to ensure that the proposed plan is carried out effectively as designed. Based on our results, limited policing of the SRC state laws in Pennsylvania high schools may occur. At a minimum, efforts to improve compliance with the state law are needed to ensure that appropriate policies are implemented and enforced correctly and consistently across the state. Presently, the responsibility for the development of SRC protocols, implementation of the state law components, and enforcement falls to individual school districts. More support from and oversight by centralized entities, such as secondary school athletic associations, boards of medicine, or both, for example, could enhance the SRC care provided to adolescent athletes statewide.

Limitations and Future Research

Although we assembled the largest sample of high school SRC protocols to date, several limitations should be recognized. First, this investigation was specific to the state of Pennsylvania, and the results are not generalizable to all high schools nationally. Also, we did not define part-time versus full-time sports medicine coverage; therefore, participants defined their own responses. Schools that responded to our participation request may have had a greater AT presence, more complete policies, or both than those schools that did not respond or participate. This should be considered when interpreting the generalizability of the study results. Furthermore, the results address only the components in written SRC protocols. The omission of specific language of the state law in a protocol may not be an oversight but rather a deliberate decision to decrease redundancy with what is already available in public legal documents. Sport-related concussion-awareness and -management strategies that are not included in the written protocol may be provided at high schools; hence, these outcomes should not be interpreted to represent policy implementation. It is also worth noting that the component analysis guide was not an exhaustive list of best practices or criterion standards for SRC management. They may be considered practice recommendations, but the true purpose of this document was to extract information for data collection. Moreover, even though the interrater reliability was sufficient for this study, more extensive assessor training exercises might have enhanced scoring proficiency. Future researchers should adopt a mixed-methods approach to investigate the congruency between written protocols and clinical practice approaches that are being implemented. Our work only captured information relative to the 2018–2019 academic year, and SRC protocols should be evaluated regularly as practice recommendations continue to develop based on the available research evidence.