Use of Patient-Reported Outcome Measures in Athletic Training: Common Measures, Selection Considerations, and Practical Barriers

Within the global health care system over the past decade, efforts have been directed at assessing patient outcomes as part of routine patient care and clinical research.^1–5 Organizations such as the Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute have highlighted the need to understand the patient's perspective on wellness and care experiences and to establish patient-oriented evidence to better inform patient care decisions.⁶ For example, through the routine and comprehensive assessment of clinical outcomes, including patient-reported outcomes, clinicians are able to identify effective treatments, treatment patterns, and areas for improvement.^1,4,5,7 Furthermore, since patient-reported outcomes are typically assessed from the patient's perspective, these outcomes are essential to delivering patient-centered, whole-person health care; establishing patient-oriented evidence that matters; and identifying the effectiveness of treatment interventions.^1,4,5,7

Patient-reported outcomes are useful for measuring what is important to patients and for evaluating patient outcomes.^7,8 The importance of assessing patient-reported outcomes in athletic training has been highlighted in the literature^8–10 and by the profession.^11,12 For example, the current editions of the “Athletic Training Education Competencies”¹¹ and the “Role Delineation/Practice Analysis”¹² emphasize the need to include patient-reported outcome measures (PROMs) as part of the entry-level competencies (effective in 2011) and continuing education efforts (effective in 2012), respectively. These PROMs are self-report surveys or questionnaires designed to capture patients' perspectives about their health status.^8,10 A variety of PROMs are available for patient care and clinical research. The 3 primary classifications of PROMs are generic, specific, and single-item measures. Generic measures are designed to be applicable to a range of health domains and to capture various dimensions of disablement and health-related quality of life.¹³ Specific measures are designed with a particular injury, illness, or body region in mind.¹³ Thus different types of measures are available, including measures specific to a body region, joint, disease, or condition. Single-item measures ask 1 question about a patient's health status and offer a quick and easy way to obtain information from the patient.¹⁴ However, single-item measures are limited in the depth of the information provided about the patient's health status.¹³

Although current efforts in entry-level¹¹ and continuing¹² education include patient outcomes assessments and the use of PROMs, research has suggested that a low percentage of athletic trainers (ATs) routinely use PROMs. Specifically, Valier et al¹⁵ surveyed a small random sample of practicing ATs and found that only 26% routinely used PROMs as part of their usual patient care. In addition to lack of time, common reasons for non-use included the lack of education and insufficient understanding of PROMs to successfully implement these tools in routine care.¹⁵ For instance, the intricate process of selecting the most appropriate PROMs for athletic health care can hinder their use in athletic training, particularly because the majority of available PROMs were developed for more general as opposed to athletic populations.^9,14,15 To our knowledge, no direct efforts have been made to identify the PROMs that are most commonly used by ATs who administer them. A better understanding of the PROMs used most often by ATs who have successfully implemented these tools in routine practice may help address the lack of education and the barriers related to PROM non-use, guide the PROM selection process, and facilitate the use of PROMs by ATs. Therefore, the primary aims of our study were to describe the PROMs commonly used by the ATs who administered them and the criteria for selecting them. Secondarily, we aimed to describe the reasons for non-use by ATs who did not use PROMs. This secondary aim allowed us to expand on previous findings¹⁵ by surveying all ATs, regardless of job setting (eg, military/government, researchers), and provide the first insight into reasons for non-use after the release of the current editions of the “Athletic Training Education Competencies”¹¹ and “Role Delineation/Practice Analysis”¹² that went into effect for entry-level and continuing education efforts in 2011 and 2012, respectively.

METHODS

RESULTS

Participant Demographics

Of the 1935 individuals who accessed the survey (response rate = 10.7%), 1732 completed the survey (completion rate = 89.5%) and were classified as either AT-USE (n = 370, 21.7%) or AT-NON (n = 1362, 78.3%). Demographics for each group are summarized in Table 1. In the AT-USE group, 67.6% (n = 250), 9.5% (n = 35), 9.7% (n = 36), and 13.2% (n = 49) classified their current position as patient care only, research only, patient care and research, or other, respectively. In the AT-NON group, 86.2% (n = 1179), 1.5% (n = 20), 1.6% (n = 22), and 10.7% (n = 146) classified their current position as patient care only, research only, patient care and research, or other, respectively.

Table 1 Participant Demographics

View Fullscreen

The χ² test results were significant for professional athletic training degree (χ² = 4.10, P = .04), highest degree earned (χ² = 108.3, P < .001), and current job setting (χ² = 101.5, P < .001), with those holding doctoral degrees (clinical doctorate, doctorate, postdoctorate), those holding professional athletic training degrees at the bachelor's level, and those working in the clinic, hospital, and military/government settings reporting the use of PROMs more frequently than other groups within the same variable, respectively. The χ² findings were not significant for sex (χ² = 1.29, P = .26) or years certified (χ² = 3.50, P = .48).

The AT-USE Results

For the AT-USE group, 51.6% (n = 191) reported using PROMs for clinical practice only, 24.6% (n = 91) for research purposes only, and 23.8% (n = 88) for both clinical practice and research purposes. Athletic trainers reported that they most frequently used specific PROMs (n = 328, 88.6%) followed by single-item (n = 258, 69.7%) and generic (n = 232, 62.7%) PROMs. Summaries of responses for specific, single-item, and generic PROMs are reported in Tables 2, 3, and 4, respectively. Of the ATs who used specific PROMs (n = 328), the knee (n = 236, 72.0%), foot and ankle (n = 212, 64.6%), and shoulder and elbow (n = 204, 62.2%) were the most commonly reported body regions (Table 2). The least common body regions for which ATs reported using PROMs were the head (n = 118, 36.0%), neck (n = 100, 30.5%), and wrist and hand (n = 99, 30.2%). The most commonly endorsed specific PROMs by ATs who used specific PROMs (n = 328) were the Lower Extremity Functional Scale (LEFS; n = 108, 32.9%); Disabilities of the Arm, Shoulder and Hand (DASH; n = 96, 29.2%); Oswestry or Modified Oswestry Disability Index (n = 80, 24.4%); and Foot and Ankle Ability Measure (n = 78, 23.8%). Athletic trainers who reported using single-item PROMs most often cited the Numeric Pain Rating Scale (n = 128, 49.6%), the Global Rating of Change Scale (n = 59, 22.9%), and the Patient Specific Functional Scale (n = 44, 17.1%; Table 3). The most frequently endorsed generic PROMs were the Short Form-12 (SF-12) or Short Form-36 (SF-36; n = 57, 36.5%) and the Disablement in the Physically Active (DPA) scale (n = 39, 25.0%; Table 4). Many PROMs received few endorsements or no endorsement at all, indicating low usage in athletic training practice (Tables 2 through 4).

Table 2 Commonly Used Specific Patient-Reported Outcomes Measures by Body Region (n = 328)

View Fullscreen

Table 3 Commonly Used Single-Item Patient-Reported Outcome Measures (n = 258)

View Fullscreen

Table 4 Commonly Used Generic Patient-Reported Outcome Measures (n = 156)

View Fullscreen

In terms of PROM selection (Table 5), the most commonly endorsed criteria were being easy for patients to understand, shown to be valid and reliable, being easy for clinicians to understand and interpret the meaning of scores and changes in scores, and being most appropriate for the types of conditions seen in the AT's practice setting. The least often endorsed selection criteria were being useful for a variety of purposes, such as research, quality assurance, and patient evaluation; suitability for electronic analysis; and seeming to be the most frequent ones used in athletic training practice. When we compared responses based on classification of current job setting, ATs who were primarily responsible for patient care rated the following criteria to be of less importance than those who were primarily responsible for research: shown to be valid and reliable (P = .03), useful for a variety of purposes (P = .02), and can be analyzed electronically (P = .03). No group differences were reported for the remaining selection criteria (P values = .07–.99).

Table 5 Athletic Trainers' Ratings of Importance of Specific Selection Criteria When Choosing Patient-Reported Outcome Measures

View Fullscreen

The AT-NON Results

Of the ATs who did not use PROMs, the most commonly endorsed reasons for non-use (Table 6) were that they require a support structure that I do not have (eg, technology, staff), take too long for clinicians to analyze/calculate/score, and take too much time for patients to complete. The least frequently endorsed reasons for non-use were that they require too high a reading level for many patients; are not sensitive to the cultural/ethnic concerns of many of the ATs' patients; and are in English, a language in which many of my patients are not fluent. When comparing responses based on classification of current job setting, ATs who were primarily responsible for patient care rated the following criteria to be of less importance than those who were primarily responsible for research: they require too high a reading level for many patients (P = .03) and are in English, a language in which many of my patients are not fluent (P = .03). No group differences were reported for the remaining reasons for non-use (P values = .06−.99).

Table 6 Athletic Trainers' Reasons for Non-Use of Patient-Reported Outcome Measures

View Fullscreen

DISCUSSION

The primary purpose of our study was to identify PROMs commonly used by ATs who administer PROMs in order to facilitate the implementation of PROMs in athletic health care for patient care and research. To our knowledge, we are the first to explicitly identify the most commonly used PROMs among ATs. Our results suggest that ATs most often reported using region-specific PROMs, followed by single-item and generic PROMs. The ATs' preference for region-specific measures was not surprising. Because specific measures were designed for a particular injury, disease, or illness, they were structured to capture small and important changes over time more easily than generic measures.^10,21 Another reason that region-specific PROMs may be used more often than other types of PROMs is that they typically focus on function, which is a major concern of athletic patients. A review of the current literature suggested that the DASH^22–24 and LEFS^25–29 were commonly used in the sports medicine community to assess self-reports of function. In our study, the DASH and LEFS were also frequently endorsed specific PROMs by ATs who used PROMs. Although these tools provide a glimpse of the patient's perspective, which should enhance patient care, their emphasis on function limits the ability to obtain a whole-person perspective on how the health condition affects the patient. Comprehensive, whole-person care requires attention to all levels of disablement, including body structure and function, activity, and participation.^7,10 Because specific PROMs tend to emphasize activity level, a generic measure should be included to complement the information identified with specific measures.^8,10,30

In our study, almost 70% of ATs who used PROMs also reported using single-item measures. Single-item measures ask only 1 question about the patient's health or perception of his or her health status, making them quick and easy instruments to administer, score, and complete.^15,16 When used in combination, multiple single-item measures can potentially capture several levels of disablement, yet they do so in less depth than multi-item measures. For example, the Numeric Pain Rating Scale, Global Rating of Function, and Global Rating of Disability capture levels of body structure and function, activity, and participation. Based on our findings, it appears that, although ATs used the Numeric Pain Rating Scale, they rarely used the Global Rating of Function or the Global Rating of Disability. Future efforts to better educate clinicians on the use of single-item outcome measures and their benefits for quick and easy outcomes assessment are needed, particularly when time is a major barrier to their use.^15,16

Our results also suggest that more than 60% of ATs who used PROMs administered generic measures. In contrast to specific measures, generic measures typically target a wide range of health domains, often assessing health-related quality of life and how the condition affects the patient from a whole-person perspective.^10,31 Thus, generic measures can often provide information beyond a person's self-report of function. In addition, because generic measures are not designed with a particular injury or illness in mind, they can be used among different patient populations and patients with different injuries.^8,10,30 This, in turn, allows clinicians and researchers to make broad comparisons across groups with various demographic characteristics.^10,32 Though generic measures offer several benefits to patient care, they may also have limitations. For example, generic measures are not developed with a patient population in mind, so they may not be as efficient in or sensitive to capturing small and important changes as specific measures are. Also, generic measures are designed to be used for a broad range of patients; therefore, some of the items may be perceived as lacking relevance to high-functioning patient populations, such as athletes.^8,10 Due to these limitations, an often recommended best practice^8–10 is to use both generic and specific PROMs to ensure that the clinician evaluates the patient from a whole-person perspective and can measure small and important changes over the course of care.

Given the potential limitations of generic measures in athletic health care, it was surprising that the DPA³³ was not more frequently endorsed in our study (n = 39, 25.0%). The DPA is a generic PROM designed specifically for highly functional patients. Lack of endorsement of the DPA may have been because it is a relatively new instrument and potentially less familiar to clinicians than the SF-12 or SF-36. Although the measure is new, recent investigators who used the DPA found it to be feasible³⁴ and reliable³⁵ in high-functioning patients. Furthermore, physical and mental composite scores have been established that may help ATs gain more insight into the patient's perspective.³⁴ As ATs continue to integrate PROMs into daily practice and clinical research, the DPA may be a useful generic PROM to consider.

A second aim of our study was to gain a better understanding of how ATs who used PROMs identified and selected them for patient care and research. Our findings suggest that criteria such as ease of understanding for the patient, demonstrated reliability and validity, appropriateness, and completion time were important factors when evaluating and selecting a PROM. The current literature on instrument selection supports these findings. For example, Kyte et al³⁶ recommended that factors to consider when selecting a PROM include its measurement properties (ie, reliability, validity, and responsiveness) and appropriateness for the patient population. Appropriateness applies to the wording in the questions and the patient burden when completing the instrument.³⁶ The criterion of time was not surprising because ATs are typically expected to provide fast-paced, high-volume patient care under time limitations. When considering the ATs' current job classification, the selection criteria for ATs who used PROMs primarily for patient care did not generally differ from those who used PROMs for research purposes. Interestingly, ATs who were primarily responsible for patient care rated the importance of a PROM being valid and reliable lower than their peers who were primarily responsible for research. This may suggest that future educational efforts should emphasize the importance of using PROMs that are valid and reliable to support global professional efforts. For example, it would be challenging to demonstrate the value of ATs and identify effective treatments and services provided by ATs without the use of valid and reliable measures.^8–10

Although gaining an understanding of the PROMs often used by ATs is important, it should be noted that the use of PROMs in athletic training was relatively minimal. Specifically, only 1 in 5 ATs used PROMs on a routine basis. This is similar to previous findings in athletic training,¹⁵ suggesting that the recommendations of the “Athletic Training Education Competencies”¹¹ and the “Role Delineation/Practice Analysis”¹² may have had little effect in encouraging the use of PROMs in athletic training practice thus far. Although the percentage was low, it is in line with results from other health care professions, such as physical therapy, occupational therapy, and speech and language therapy.³⁷ When considering whether certain demographic factors influenced the use or non-use of PROMs in athletic training, it appears that education and work setting may have affected the use of PROMs. Our findings suggest that individuals who had doctoral training were more likely to report using PROMs. This may speak to the notion that additional education may influence whether an AT will use PROMs. This is reasonable considering that one of the major reasons for non-use was lack of training as reported by ATs¹⁵ and physical therapists.¹⁴ In addition, it appears that the work setting can affect the use of PROMs; individuals working in clinic, hospital, and military/government settings were more likely to use them.

Beyond potential differences based on demographic factors, an understanding of barriers related to the non-use of PROMs is essential in guiding future efforts for increasing the use of these instruments. As in most health care settings, the lack of time to complete, score, and interpret PROMs is a major barrier.^37–40 Technology may help to alleviate these time constraints.³⁹ For example, resources such as Web sites⁴¹ and electronic medical records⁴² can facilitate the completion and scoring of PROMs, thus reducing the time burden. Also, efforts have been focused on using computer-adaptive testing methods for PROMs,^43,44 reducing the overall number of items patients complete, and further limiting the time needed to administer the instrument and calculate and interpret scores. Future educational efforts should focus on identifying and developing ways to use technology to encourage more routine use of PROMs. Additionally, researchers in athletic training should consider using computer-adaptive testing methods.

Our findings also suggest that the lack of organizational infrastructure is a major barrier to the routine use of PROMs. Previous investigators in a variety of health care professions,³⁹ including rehabilitation health sciences (eg, physical therapy, occupational therapy, and speech and language therapy)³⁷ and palliative care,³⁸ have highlighted the importance of a supportive organizational infrastructure in encouraging PROM use. Essential components of organizational support for PROM use include policies to guide PROM use,⁴⁰ clear guidelines for implementing PROMs,³⁹ managerial involvement and support (eg, managerial appreciation for the extra effort, involvement in the implementation process),^38,40 adequate resources (eg, training, technical support, statistical support, data interpretation),^37,39 and additional administrative support (eg, a PROM coordinator).³⁸ In our study, we found that ATs practicing in clinics, hospitals, and military/government settings, which likely have more robust and defined organizational infrastructures, were more likely to use PROMs than ATs working in more traditional settings, such as secondary schools, that may have limited resources. Because most ATs do not work in traditional medical settings or under the medical model, professional efforts are needed to identify ways of providing clinicians with more support for PROM use. For example, professional organizations, such as the NATA, may consider providing resources such as training, technical support, statistical support, and data interpretation to assist ATs in the use of PROMs. In addition, efforts by professional organizations to develop best practices for PROM use may offer support similar to that supplied by more formal organizational policies. Future efforts by the athletic training profession should aim to identify strategies that help ATs optimize organizational infrastructure and support to encourage the use of PROMs in patient care.

One limitation of our study was the relatively low response rate (10.7%). However, similar response rates have been reported in other athletic training studies.^45–47 To our knowledge, our results reflect the highest response rate of ATs concerning their practice patterns and the use of PROMs in athletic health care. Also, the publicly available demographic information of NATA members (eg, sex, job setting, highest level of education, district)^48,49 was similar to the percentages observed in our sample, supporting the representativeness of our sample and the generalizability of our findings. Lastly, the final response size of 1784 exceeded the threshold needed to provide results that were accurate at a 99% confidence level and ±3% margin of error. Thus, our results likely provide a better estimate of the population value than those of Valier et al,¹⁵ who reported a 31% response rate, though their study was powered for a 40% response rate, 95% confidence level, and 64% margin of error. For these reasons, we believe our results add meaningful information to the current literature and provide beneficial insights into the use and non-use of PROMs in athletic training. Another possible limitation is that although we made efforts to be inclusive and comprehensive in our list of PROMs, some available measures were not included as options in our survey. Even though respondents were able to cite PROMs not included in our study by typing in entries, some instruments may have been missed. Lastly, we did not operationalize the term routinely when asking the participants whether or not they routinely used PROMs for clinical practice or clinical research, which they may have interpreted differently. Despite these limitations, we believe the findings from our study provide important information on the current use of PROMs among ATs.

CONCLUSIONS

We aimed to describe PROMs commonly used by ATs who administered PROMs, the criteria by which ATs selected PROMs, and the reasons for non-use by ATs who did not administer PROMs. Although the overall use of PROMs in athletic training remained low, those who did use PROMs reported administering a variety of different types. Generally, specific PROMs, such as the Foot and Ankle Ability Measure, LEFS, and DASH, were the most commonly reported PROMs used by ATs, but more than half also indicated using single-item and generic measures. When selecting a PROM, ATs often consider the time burden for the patient and the clinician and the measurement properties of the PROM. To encourage greater use of PROMs in the athletic training profession, efforts are needed to address common barriers, such as lack of time, and to identify effective strategies (eg, use of multiple single-item measures, leveraging available technology) to help ATs implement PROMs more routinely into patient care and research.