Improving Athletic Trainers’ Knowledge of Clinical Documentation Through Novel Educational Interventions: A Randomized Controlled Trial

Design

This paper is part of a larger sequential explanatory mixed methods project that explored ATs’ knowledge of clinical documentation before and after completing an educational intervention. We followed a previously published protocol for a randomized control trial to evaluate the effectiveness of an interactive versus passive learning mechanism.¹³ The full description of the development of the clinical documentation educational interventions was published by Welch Bacon et al.¹¹ Additionally, to understand ATs’ experiences with the educational materials, we used the consensual qualitative research (CQR) approach.^14,15 The CQR approach has been commonly used in athletic training research to explore perceptions while minimizing biases through the use of multiple researchers and a systematic, transparent approach throughout data analysis.^{4–6,9,16,17}

Intervention Materials and Instruments

The online educational interventions and knowledge assessment were hosted in Qualtrics. The primary research team (T.M.K., S.L.N., C.E.W.B.) has an extensive record collaborating on projects regarding clinical documentation. Therefore, we developed 2 online educational interventions: (1) an interactive, guided platform and (2) a comprehensive list of reading materials.¹¹ The interactive personalized learning pathway (PLP) consisted of 8 separate modules with short videos, readings, reflection prompts, quizzes, and a space within the platform to take notes. The passive reading list included 14 hyperlinked resources that were grouped into 8 topic areas that matched the PLP modules. Three subject matter experts provided feedback on the content of the educational interventions, and minor adjustments to formatting were made before piloting with 20 ATs. Based on pilot feedback, we added the anticipated time to complete each PLP module, a progress bar, and a back button. For the PAS reading list, we reorganized materials into specific categories and added headings to visually separate materials on the web page. The detailed description of the development, validation, and piloting of the educational interventions are published in Welch Bacon et al and are outlined in Figure 1.¹¹

View Figure

• Download as Powerpoint
• Download Figure

To assess ATs’ knowledge of clinical documentation, we developed an instrument based on published research and best practices.^2–7,9,18 The same subject matter experts reviewed the 35 multiple choice items and commented on item clarity, accuracy, and comprehensiveness. We made minor revisions for clarity to 10 items based on their feedback. To pilot the instrument, we administered it before and after the 20 ATs piloted the PLP (n = 10) and PAS (n = 10) interventions. Item mean change scores (preintervention and postintervention) indicated improvement in pilot ATs’ knowledge after completing 1 of the interventions (PLP: M = 4.9% ± 14.8%, PAS: M = 4.0% ± 13.5%). Item scoring was also evaluated within and between groups, and the instrument was deemed to have good discrimination. Additionally, pilot participants’ comments on the items were collected after completing the postknowledge assessment, which resulted in minor changes for clarity to 11 items, and 1 item was deleted. Thus, the final instrument included 34 items. Scores from piloting were not included in the study findings.

To qualitatively evaluate the educational interventions and participants’ experiences, the primary research team (T.M.K., S.L.N., C.E.W.B.) developed interview guides that built on prior research on documentation.^4,9,16,17 An internal auditor with extensive qualitative research experience assessed the guides, provided feedback, and determined they were clear and captured the objectives of the study. The guides contained core questions regarding ATs’ prior educational preparation on clinical documentation, their knowledge and confidence regarding clinical documentation, and their experience with the educational materials (Table 1). The full interview guides were published by Nottingham et al.¹⁹ No modifications or clarifications were necessary after the initial interview within each group; therefore, these interviews were included in the analysis.

Table 1. Summary of Core Interview Items for the PLP and PAS Participants^a

View Fullscreen

Study Procedures

Figure 2 illustrates participant recruitment, enrollment, completion of the interventions, and data analysis procedures. Clinical documentation is a required component of ATs’ clinical practice; therefore, all certified members of the NATA across all clinical practice settings, excluding full-time educators, were invited to participate using the NATA survey system. Certified ATs (n = 18 979) were emailed a description of the study and a hyperlink to enroll. A total of 582 ATs started the enrollment survey, and 561 ATs submitted the survey, of which 37 ATs did not meet the inclusion criteria or did not consent. Therefore, eligible ATs who enrolled in the study (n = 524) were randomly assigned to groups (PLP = 178, PAS = 176, CON = 170) and took the online knowledge assessment. Participants did not receive feedback regarding their performance on the knowledge assessment at that time. They were then given a separate hyperlink to their educational intervention (CON, PAS, PLP) and 4 weeks to complete the educational intervention (or no intervention for the CON participants), and the PLP and PAS participants were emailed reminders at the 2- and 3-week marks. After week 4, participants took the knowledge assessment again, which concluded the study. The PLP and PAS participants who indicated willingness to be interviewed at the end of the postknowledge assessment were contacted 1 week after the study concluded. One researcher (S.L.N.) completed all interviews on a web-based teleconferencing platform (Zoom), and interviews were transcribed verbatim. A research assistant helped with reviewing the deidentified transcripts for clarity before analysis.

View Figure

• Download as Powerpoint
• Download Figure

Data Analysis and Management

Quantitative Knowledge Assessment Scores Preintervention and Postintervention

At the onset of the study, ATs self-reported that they were comfortable (M = 4.2/5.0 ± 0.6) and competent (M = 4.0/5.0 ± 0.7) regarding their knowledge of clinical documentation. Group demographic characteristics (age, years certified, gender, race or ethnicity, employment setting, and documentation approach) did not differ between groups. Also, no statistical differences were found between groups in the preknowledge assessment scores. We identified a significant group × time interaction (F_2,157 = 15.30, P < .001; partial η² = 0.16). Post hoc analysis of mean change scores revealed the PLP (M = 8.82% ± 7.94%, P < .001) and PAS (M = 5.00% ± 8.82%, P = .012) groups improved significantly more than the CON group (M = 1.18% ± 6.47%). Additionally, the PLP group exhibited a greater mean change than the PAS group (P = .049).

Descriptive analysis of individual items demonstrated that more participants in the PLP group correctly answered items related to details required in treatment logs (100%, n = 39/39; PAS = 86.4%, n = 38/44; CON = 83.1%, n = 64/77) and inclusion of a discharge note at the completion of the patient’s care (94.9%, n = 37/39; PAS = 93.2%, n = 41/44; CON = 75.3%, n = 58/77). Collectively, participants scored poorly on the postknowledge assessment for items related to appropriate use of abbreviations (44.4%, n = 71/160), strategies for maintaining system security (13.1%, n = 21/160), the definition of a personal health record (48.1%, n = 77/160), and the description of health information technology (41.9%, n = 67/160). Table 4 includes additional descriptive data for select items from the postknowledge assessment and related interview quotes. Figure 4 illustrates the postknowledge assessment mean scores when grouped by topic area and sample interview quotes regarding the key content.

View Figure

• Download as Powerpoint
• Download Figure

Table 4. Sample of Athletic Trainers’ High- and Low-Performing Topic Areas Regarding Clinical Documentation

View Fullscreen

Qualitative Theme: Knowledge of Clinical Documentation

A theme regarding ATs’ knowledge of clinical documentation emerged, specifically about their perceived confidence in their knowledge and skills and key content they identified from the educational interventions. Categories and frequency counts are also presented in Figure 3.^14,15

Areas of Strengths Regarding Clinical Documentation

Participants across groups were knowledgeable about professional standards and reasons for documenting patient care. They also correctly answered items related to the details included within patient cases (eg, information within SOAP note sections, how to update a note) and qualitatively reported their overall confidence in documenting patient encounters. This aligns with previous reports that ATs across practice settings document for similar reasons (eg, professional responsibility, legal protection, tracking patient progress) and commonly use the SOAP note format.^3–5,16,17 Participants in the current investigation also identified that the materials on strategies to streamline and cross-check documentation practices were helpful. For example, Linus (PLP) valued learning about chart auditing, and Rinna (PAS) appreciated clarification of what details to document. The NATA Best Practices in Athletic Training Documentation outlines many recommendations, but it appears that ATs want more specific examples and strategies of how others are implementing these recommendations.²

Another noteworthy area was ATs’ knowledge of patient-reported outcomes (PROs). Although we did not evaluate ATs’ use of PROs in this study, participants correctly identified general versus specific PROs and how they differed from clinician-reported outcomes, which is an improvement in ATs’ familiarity with PROs compared with a prior study.²⁰ This is unsurprising, as the athletic training profession continues to evolve, but guidance for administering PROs as routine practice is still needed. For example, Jenny (PLP) and Derek (PLP) reflected on a module in which a PRO was seamlessly integrated into the patient case and highlighted the importance of capturing the patient’s perspective. Including PROs in all patient scenarios within professional education and CE programs is one way to promote PRO use and its role in documenting a complete patient case. Supervisors should also consider establishing expectations for their staff to implement PROs during patient care, and using chart audits and peer review within the workplace may improve quality and compliance with documentation.¹⁶

Regardless of completing the passive reading list or interactive online modules, ATs found the materials to be valuable and significantly improved their knowledge compared with the control group. Furthermore, PLP participants demonstrated greater improvement than PAS participants, which suggests the PLP format is an effective way to deliver information in a cost-effective, convenient, and learner-focused method.^11,13 It is likely the variety of media, such as reflection prompts, graphic lecture style, or videos of practicing clinicians describing their documentation strategies, increased the relevancy of the information and kept the participant engaged. Since ATs have reported that interest and accessibility drive their decisions on CE programs to complete, the PLP active learning format may help keep ATs focused to better facilitate knowledge acquisition, particularly since they are already taxed with balancing CE among other work and personal responsibilities.^21,22

Areas for Improvement Regarding Clinical Documentation

It is not surprising that the control group, who did not complete an educational intervention, scored the lowest and exhibited no significant improvement in their knowledge scores. Although the PLP and PAS groups did exhibit significant improvement in their knowledge, knowledge gaps still exist. For all ATs, the most pressing need is to address strategies to promote legal protection and data security. In the current study, ATs scored poorly on items regarding safely securing electronic and paper-based patient health information and ensuring email communication is saved to a patient file and deleted off a server. Additionally, ATs in our study reflected on possible Health Insurance Portability and Accountability Act (HIPAA) and Family Educational Rights and Privacy (FERPA) violations with commonly used paper sign-in sheets or forgetting that text messaging was not a secure method for communicating health information. Although legal protection may be a primary reason that ATs document, their performance on these items suggest they may be less protected than they realize.^3,4 Therefore, engaging in periodic CE on clinical documentation, auditing a random selection of patient charts, and reflecting on current behaviors would be beneficial for ATs’ legal protection throughout their career.

In the current investigation, ATs across groups also scored poorly on items related to health information technology and estimating the value of an AT. Interestingly, these topics were also identified during interviews as key content learned within both educational interventions. The Health Information Technology for Economic and Clinical Health Act calls for the use of technology such as EMRs.²³ As the athletic training profession continues to respond to changes in technology and the health care system, it is important for ATs to also adopt and use EMRs.²⁴ Athletic trainers have previously reported challenges with internet connectivity, firewalls, or administrative support regarding the use of EMRs; however, emphasizing how EMRs can help limit HIPAA or FERPA violations may garner administrators’ support.⁹ Documenting in an EMR at the point of care ensures it is completed in a timely manner and can also improve documentation quality to better capture the athletic training services provided and estimate the cost of those services to demonstrate the value of employing an AT.²⁴ Although ongoing CE in this area is still needed, our educational interventions appear to be a great starting place with practical application of health care economics (eg, showing return on investment) and health information technology (eg, using EMRs, improving documentation compliance and quality) to promote the vitality of the profession, which are also components of the prioritized athletic training research agenda.²⁵

Limitations and Future Research

Findings from this study may be limited in transferability based on participant characteristics. We sent an email to the NATA membership to broaden participation across employment settings, years of experience, and other member characteristics; however, it is possible that more members who were already confident in their knowledge of documentation elected to enroll in the study. Whereas many participants dropped out before completing the preknowledge assessment, this attrition is not surprising. It is possible participants did not want to take the lengthy knowledge assessment or that the Category D Board of Certification continuing education units (CEUs) were less of an incentive to complete all study components. It is also likely the anticipated time requirement (PLP = 4–5 hours, PAS = 2–3 hours) affected the completion rate, but these educational resources were intended to be comprehensive, accessible in 1 place, and could have been completed at any time over a 4-week period. Whereas the completion rate was not as high as desired, the study was adequately powered even without Category A CEUs, and future completion of these educational interventions would not require completion of a preknowledge nor postknowledge assessment. Therefore, we believe these comprehensive educational interventions on clinical documentation are valuable to practicing clinicians. Furthermore, the PLP has been approved for Category A CEUs starting January 2024, which may provide additional incentive to complete it.

Moving forward, these educational resources on clinical documentation will require updating in the future as best practices shift or as policies change. Although we observed improvements in PLP and PAS participants’ knowledge of clinical documentation, we did not track their behaviors nor specific changes made to their documentation practices after the intervention. Whereas improving knowledge of clinical documentation is important, consideration of how to promote knowledge retention and knowledge translation is also needed. Future researchers should explore the learners’ perceptions of self-directed versus interactive mechanisms and whether different formats affect knowledge retention and translation to clinical practice.