Organizational Expectations Regarding Documentation Practices in Athletic Training

Research Design

We used a concurrent mixed-methods study approach using a combination of web-based surveys and follow-up interviews. This design allowed us to draw from the strengths and minimize the weaknesses of both qualitative and quantitative design types to contextualize the meaning behind the data.^14,15 Using a mixed-methods approach, we took the quantitative data of the survey and amplified the values using the voiced collective experience of current supervisors. The targeted approach allowed us to elaborate and uncover areas of content not represented in the use of 1 tool alone.¹⁶ Before data collection, the risks and benefits of the study were explained to all potential participants, and they provided consent to participate. The Indiana State University Institutional Review Board deemed our study exempt.

Participants and Setting

We created an internal database of supervising collegiate and university athletic trainers (ATs) and their emails (N = 1107) in Divisions I, II, and III of the National Collegiate Athletic Association (NCAA) using publicly available webpages. Using the publicly available contact information, we selected individuals with titles such as head AT, director, manager, supervisor, or coordinator or individuals who otherwise indicated they oversaw the services of other ATs and sent emails through Qualtrics software (Qualtrics LLC). Of the 1107 possible participants, 106 accessed the online survey (9.58%), 5 reported they were not supervisors, and 5 did not consent. Of the 96 individuals who identified as eligible supervisors and consented to participate, 64 (66.7%) completed the survey in its entirety (survey participants’ age = 43 ± 11 years; years of experience as an AT = 21 ± 10; years of experience as a supervisor = 12 ± 10). Twelve participants elected to participate in the follow-up interview (age = 35 ± 6 years; years of experience as an AT = 14 ± 5; years of experience as a supervisor = 8 ± 5). Table 1 describes the survey demographic characteristics of all participants, and Table 2 depicts specific characteristics of the interview participants.

Table 1. Survey Demographics (n = 64)

View Fullscreen

Table 2. Interview Demographics (n = 12)

View Fullscreen

Instrumentation

Our research team created a 24-item survey instrument based on the study’s specific aims using a variation of close-ended Likert-scale questions and open-ended questions. We developed questions deductively based on findings from previous literature on documentation practices in athletic training.^9,10,12,13 We asked these participants to answer questions regarding their expectations, organizational policies and procedures, and strategies to encourage their staff to produce high-quality medical documentation. The questionnaire addressed 1 screening item; demographic items (9); medical documentation policy, preparation, and expectations (11); perception of employee competence (2); and interest in participating in a follow-up survey (1). We measured employee competence by asking for perceived confidence in policy and procedure clarity, thoroughness, and confidence in employee ability to generate complete and accurate patient care documentation. Employee competence and clarity and thoroughness were measured on a 6-point Likert scale: strongly disagree (1), disagree (2), slightly disagree (3), slightly agree (4), agree (5), and strongly agree (6).

We validated the survey using a content validity index process as described by Polit and Beck.¹⁷ We sent the survey to a panel of 5 content and/or methods experts in survey creation, medical documentation, and/or supervision of ATs. We instructed each expert to complete a content analysis rubric for the survey, with options to select Needs Revision or Sufficient as Written and a space for additional comments to address either category. The rubric used a numeric scale to rate the relevance and clarity of each item rating, including not relevant (1), item needs some revision (2), relevant but needs minor revision (3), and very relevant (4). These values were then averaged to create a total relevance and clarity score. To deem the survey valid, we required each item to have a mean agreement of 75% for relevance and clarity. We analyzed and synthesized all feedback and required no additional rounds of review. No items needed revision or removal from the tool. The tool’s scale-level content validity index score is 0.93, demonstrating excellent content validity.¹⁸

We used semistructured interviews to gather additional insight into the effectiveness of communication, dissemination of information, and quality of existing policies and procedures based on the quantitative information collected in the survey. The research team created an interview protocol with 7 primary questions and potential follow-up questions based on the study’s specific aims and past literature using the same process described above.¹⁰ Two individuals with experience in qualitative research and medical documentation reviewed the interview script for face validity. The interview protocol required no additional revisions. The primary investigator practiced the interview with 3 eligible individuals to assess timing and question sequencing; their data were not included in the analysis. Table 3 describes the questions used in the interview.

Table 3. Semistructured Interview Protocol^a

View Fullscreen

Procedures

Our study used email recruitment methods to garner participants. We sent initial emails to all potential participants, including the study’s purpose and a survey link. We sent reminder emails weekly during the 6-week data collection period. Once participants indicated consent, they spent approximately 28 minutes completing the survey. We secured all survey responses within Qualtrics and deidentified them before downloading and saving them to a secure cloud storage site.

The last question on the survey provided participants the opportunity to indicate interest in participating in a follow-up interview. Participants who indicated interest were directed to an additional survey to submit their contact information. The principal investigator (PI) conducted all interviews, and they lasted, on average, 25 minutes. After each interview, the PI saved the deidentified transcript and audio file, using a pseudonym, to a secure cloud storage site. We sent participants a copy of their transcribed interview to ensure accuracy and instructed them to clarify and address potential concerns within 14 days. If participants did not respond within 14 days, we used their original transcripts for analysis. Only 1 participant responded requesting minor copyedits. Saturation was achieved when the PI no longer heard new core ideas during interviews. The data analysis team (3 members of the overall research team) discussed field notes and PI observations before concluding that data saturation had been achieved.

Data Analysis, Researcher Reflexivity, and Trustworthiness

We used descriptive statistics to examine demographic variables, including participant age, university setting, years of experience as a certified AT, years of experience as a supervisor, and the number of staff under supervision for the survey and follow-up interview.

Incomplete data sets are common in survey research, as this research is voluntary, and discontinued participation is the participant’s right, as stated in the Belmont report.¹⁹ Because the number of respondents varied per question, we used partial data analysis techniques to manage missing data.²⁰ These techniques are used throughout the literature to account for potential bias and the lack of randomness with missing data.^13,21 We set the significance level a priori at P < .05. We analyzed all quantitative data using SPSS (version 27; IBM Corp).

We used the consensual qualitative research tradition to develop codes from the interview transcripts due to its previous use in athletic training research and medical documentation and its aptitude for in-depth analysis.^10,22,23 The data analysis team consisted of 3 individuals (J.S.D., M.J.D., L.E.E.) who ranged from novices to experts in the consensual qualitative research tradition. The 4-step process began with an inductive approach, reviewing the first 5 transcripts.²² After, each member of the data analysis team independently reviewed these transcripts to create a domain list reflective of the data.²² The data analysis team then met and developed an initial codebook with domains and categories.²² The codebook was then applied to 2 of the original transcripts and 3 new transcripts to ensure an accurate representation of ideas.²² The data analysis team met again to confirm that the original codebook was still accurate.²² After, the PI applied the consensus codebook to all the transcripts.²² The research team met for a final time to confirm that the codes were applied correctly across all transcripts. The PI then created a cross-analysis of all participant interviews.²² The research team used the cross-analysis to ensure that all quotes under each domain and category had no outliers and that the placement of the quotes within each category was appropriate.²²

After completing all steps, the data analysis team sent the interview script, consensus codebook, cross-analysis report, and coded transcripts to an external auditor (E.R.N.) to explore blind spots or selective perceptions or biases of the data analysis team during the coding process.²² We made no changes from the review. Triangulation of the data was ensured by comparing the domains from the qualitative data with the quantitative data of the survey to examine similarities and create a comprehensive representation of the results. Research reflexivity is an expected factor in qualitative research. The PI acknowledges the potential impact of their previous experiences, assumptions, and beliefs in medical documentation while conducting interviews.²⁴ We established trustworthiness by using multiple-analyst triangulation, member checking, and internal and external auditing.

Framework

Participants described having a philosophy around orienting new and existing employees, communicating policies and procedures, holding employees accountable by using different strategies, and identifying a clear purpose that served as a framework to organize their staff development. Approaches that supported successful documentation practices included annual training/development and education, individual meetings, frequent documentation auditing or review, and an emphasis on continuity of care and advocacy for additional resources.

Limitations

The limitations domain included organizational concepts relative to barriers in creating high-quality documentation.

Reasons, Criteria, and Mechanics

The Strategic Alliance’s Athletic Training Research Agenda was developed to improve patient care and advance the profession, and improving medical documentation compliance and quality is listed within the Health Information Technology priority.²⁵ Research regarding athletic training medical documentation continues to grow. In our study, the reasons supervisors cited for creating medical records, selecting criteria for which encounters to record, and choosing mechanisms for documenting were consistent with past literature,^8,9,13 highlighting a similar level of understanding between the ATs functioning as employees and those functioning in supervisory roles. These reasons included continuity of care and delivery, communication, “cover yourself” and legal implications, and demonstrating value. Despite similar levels of understanding between supervisors and employees, the findings from our study imply that the execution of enforcing quality documentation practices is variable between different organizations.

Policy and Procedure Creation

The NATA Best Practice Guidelines serve as a reference for the importance of documentation, key terminology, understanding of local and federal rules and regulations, and setting-specific considerations.⁷ These guidelines do not dictate specific standards for documentation mechanics, inclusion criteria, timeliness, or accountability expectations. It is worth noting that the lack of standardization does allow ATs to be flexible in creating policies and procedures that meet local organization needs. The solution to inadequate medical documentation in athletic training may not be complete standardization but more specific guidelines and encouragement for individual organizations to take responsibility and ownership in creating and upholding their own policies and procedures.

In our study, no supervisors commented on consulting with legal counsel when creating their policies and procedures regarding medical documentation. Individuals described professional education and socialization, life and clinical experience, trial and error, and quality improvement (QI) projects as sources for developing policies. Supervisors should coordinate with risk management experts when crafting policies and procedures for emergencies and daily operations because unexpected or unwanted outcomes may lead to litigation if an appropriate process is not in place.^26,27

Our study showed similar results as those reported in previous studies^28,29 regarding the lower prevalence of formal policies and procedures, with fewer than half of supervisors reporting having formal written organizational documentation policies, procedures, and time expectations for recording encounters. One study examined written and nonwritten organizational practices and reported that only half of the secondary school ATs were using risk management practices to write their policies and procedures.²⁸ Another study found that policies and procedures regarding administrative responsibilities, including medical documentation, were operational and written only 64.3% of the time; 20.8% indicated that they were operational only, 0.9% noted that they were written only, and 14.0% indicated that they were neither written nor operational.²⁹ They also found that only 35.3% of collegiate ATs had guidelines for periodically reviewing their policies and procedures.²⁹ Potential strategies for improving policy and procedure knowledge retention and enforcement include developing clear and concise expectations and instructions and reviewing materials at least annually and as often as necessary.³⁰

Accountability, Education, and Training

In addition to considering risk management when creating policies and procedures, health care departments should also consider managerial, organizational, and clinical approaches to investigate multiple perspectives on service effectiveness and efficiency. Known as clinical governance, this approach provides a framework that encourages cohesiveness among all personnel for achieving goals related to improving quality and patient care safety in a way that is practically manageable for supervisors and executable by employees.³¹ An example of this type of project in previous literature included using patient-rated outcome measures to assess the impact of health care interventions on patients and guide resource allocation.³² The cohesiveness of clinical governance can help ensure systemic dissemination of concepts, timely interventions for performance gaps, and consistent standardization in record quality.³³ To address poor performance, individuals must communicate feedback with clear expectations to support the rationale. Directness creates the basis for accountability, which is essential for organizational change and transformation. Past research supports 6 elements for supervisors in health care to implement successful change: a compelling vision with a sense of urgency, meaningful and realistic plans understood by stakeholders and employees, teamwork and empowerment, effective 2-way communication, expanding on previous success to create momentum, and strong and effective leadership.^34,35

Accountability and lack of accountability were significant findings within our results as well as previous research in medical documentation practices in athletic training.^10,13 Evidence from our study identified accountability strategies that included annual training and education, weekly injury report checks, 1-on-1 meetings, and medical documentation review and auditing. However, some evidence suggests that employees may need more than resources, periodic tips, and refreshers to inspire sustainable improvement.⁸ Various tools have been used among health care professionals to inspire such improvement, and these tools often need to reflect the profession’s and organization’s services and responsibilities. For example, self-audits and peer audits effectively improve documentation quality.^36,37 Although these internal audits may be an effective way to engage staff in mutual accountability, clinical auditing must be completed with a high level of knowledge on accuracy, availability, completeness, relevance, reliability, timeliness, and validity to produce higher-quality medical records.³⁷ Supervisors should consider exploring different auditing strategies and selecting interventions based on the individual organization’s needs, building the process into their documentation policies and procedures to enforce standards and expectations.

Implementation of the new Commission on Accreditation of Athletic Training Education professional program Curricular Content Standards around QI and health care informatics essentially demands improved education and exposure during professional preparation and can serve to improve documentation practices across the profession.³⁸ Recent literature suggests that academic electronic health record systems may assist in training athletic training students. These systems use activities such as educational tasks, documentation projects, critical reviews of standardized patient cases, and assessments of patient care data for QI efforts to improve competency in informatics and documentation skills.³⁹ Moreover, these practices also offer an opportunity to evaluate practice and improve patient safety, and engaging in these learning activities when preparing future ATs will likely improve the skills of documenting and the culture around documentation in the coming years.

Prioritization

Athletic training literature has referenced lack of time as a barrier for a multitude of practice behaviors, including medical documentation.¹⁰ Our study echoed these sentiments, specifically with medical documentation, as previous research has suggested that it is not a lack of time but a lack of priority, a lack of knowledge, and a lack of incentive.^9,10 The rationale of prioritizing patient care and supporting high caseloads or event coverage is echoed in our study alongside other athletic training medical documentation studies.^8,10,13 Other health care professions are often beholden to insurance companies and the Centers for Medicare and Medicaid Services; to get paid, one must document.⁴⁰ It may be because of the historic nature of athletic training as a cost-containment effort⁴¹ that the athletic training profession has failed to incentivize documentation, but it is truly indiscernible when and where the shift in deprioritizing documentation occurred in athletic training. Although medical documentation is required for clinical practice in health care, many athletic training organizations are still falling short.

Evidence from our study and past research may suggest that the common culture of athletic training places patient care and constant availability above all else, disrupting the necessary time and energy to dedicate to other roles and responsibilities. It is evident that a cultural shift is needed to comply with best practices in order to optimize time and reduce medical errors for legal and patient safety reasons. High-quality documentation ensures the completion of many essential components of care. These components include, but are not limited to, a comprehensive evaluation, consideration of differential diagnoses, timely and appropriate treatment of potential or active life-threatening conditions, and creation of clear care plans noting follow-up testing and referral to other providers as needed. A choice to prioritize other roles and responsibilities above medical documentation is a choice, albeit not an intentionally malicious choice, to deprioritize patient safety.

Limitations and Future Directions

Although survey research can reach many participants, there is a risk for self-selection bias⁴² and sampling bias because we only surveyed ATs working as supervisors within NCAA-affiliated schools. Our population sample may have been more open to talking about their documentation practices and thus could influence the generalizability of the results. Our lower response rate could also be attributed to this limitation, as 95% completed the survey through demographics, but only 68% completed the tool in its entirety.

Future intervention research in medical documentation should evaluate the effectiveness of different strategies for improving the overall quality of documentation and frequency of documentation completion. Further studies could take the form of research commentary or implementation science, such as QI projects.^43,44 Quality improvement projects might illuminate the process of creating policies and procedures or explore the process of improving medical documentation locally. Potential areas of improvement might include enhancing discharge⁴⁵ and diagnosis⁴⁶ processes, item inclusion criteria,^47,48 and holistic care approaches.⁴⁹ Additional research could also evaluate the differences and similarities between documentation practices across other health care professions, levels of education, and training completed to meet organizational standards and foster interprofessional collaboration and communication.