Patient-Reported Outcomes Measurement Information System Physical Function Item Bank, Version 1.0: Physical Function Assessment for Athletic Patient Populations

As athletic trainers (ATs) embrace the global health care approach of evidence-based decision making,^1–4 identifying outcome-assessment instruments that are appropriate for their clientele becomes paramount. Specifically, ATs must adopt patient-reported outcome (PRO) instruments that can efficiently and effectively measure the entire spectrum of a patient's abilities. Because ATs practice in a variety of settings, the physical functioning of clientele can vary significantly. Therefore, PRO instruments must enable ATs to capture all levels of functional ability from the acutely injured recreational walker (low functional ability) to the healthy, injury-free, elite-level athlete (high functional ability).

Unfortunately, the assessment tools currently available to ATs have limitations.^2,5,6 First, many of the assessment instruments used in athletic training environments are based on classical test theory, which limits the adaptability of the instrument and forces patients to answer lengthy questionnaires, regardless of their current physical function levels.^3,5,7 For example, a patient who is using crutches during the early stages of rehabilitation will answer questions related to walking up stairs and jogging. Second, researchers have recommended that clinicians administer both generic and region-specific PRO instruments throughout the course of a patient's treatment protocol to accurately assess all dimensions of health-related quality of life.⁶ So, instead of 1 lengthy questionnaire, clinicians are now asked to administer a number of instruments, further contributing to time constraints in the clinic. Ultimately, administering lengthy questionnaires that cannot adapt to a patient's level of functional ability or individual injury places an excessive burden on both the patient and the clinician.⁵

Finally, although many of the commonly used PRO instruments have at least some supporting evidence for validity, their ability to assess individuals representing the top tier of functional ability (eg, elite athletes) has not been established. Item difficulty, item discrimination, and ceiling effects are the basis of all measurement limitations associated with traditional PRO instruments.^3,5,7 For example, the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36) is a popular questionnaire designed to measure a patient's general health status based on both physical and mental constructs.⁸ However, 37% to 72% of the general population will reach the ceiling of the functional dimensions in this test,⁹ making it difficult for clinicians to apply the results to a sport-specific population. In other words, a perfect score on the SF-36 does not necessarily mean an athlete is healthy and ready to return to competition; rather, it merely indicates that the athlete's level of physical function is comparable with that of the general population. Only by completing additional evaluations will a clinician be able to determine a more accurate functional assessment.

However, a potential solution to the measurement concerns regarding athletic training clientele and traditional PRO instruments is available. Computer-adaptive testing, grounded in the principles of modern measurement theory, offers the ideal platform through which ATs can use PROs. Unlike classical test theory, computer-adaptive tests use psychometric properties of item response theory to rank order unidimensional items based on difficulty.^10,11 Essentially, the difficulty of the items and the ability of the patients can be placed on a common metric to allow for a better understanding of the capabilities and precision of an instrument. When these tests are administered using computer-adaptive platforms, unnecessary or redundant items are eliminated and fewer questions are then needed to capture a patient's current health status, improving test accuracy and alleviating patient and clinician burden.¹² Furthermore, because these computer-adaptive tests can draw upon large item banks, they offer the potential to minimize ceiling effects.¹³

A specific example of a PRO instrument that has the potential for use by ATs is the Patient-Reported Outcomes Measurement Information System (PROMIS),¹⁴ a project of the National Institutes of Health that began in 2004. This project was designed to create universal computer-adaptive item banks that could assess a number of health care domains, including pain, fatigue, emotional distress, physical function, and social function.¹⁵ To formulate these item banks, PROMIS investigators drew individual questions from a number of well-established paper-and-pencil PRO instruments and applied the properties of item response theory.¹⁵ Researchers¹⁶ can now use PROMIS software to create computer-adaptive tests that mathematically administer questions from larger item banks depending on the patient's current health status.

Those ATs seeking to monitor the effectiveness of a treatment protocol may find the PROMIS physical function item bank of particular interest. This item bank contains 124 items (ie, questions) designed to measure an individual's ability to perform physical tasks ranging from low-level skills (eg, activities of daily living) to more difficult skills requiring mobility, strength, and endurance.¹⁶ Preliminary validity evidence for the PROMIS physical function item banks has been established in general populations compared with the SF-36 (Pearson r = 0.88),^16,17 as have precision and reliability for lower extremity injury, central body function, and activities of daily living.^17–19 More recent research²⁰ has demonstrated strong convergence correlation of the PROMIS physical function item bank with the International Knee Documentation Committee Subjective Knee Form (r = 0.75) and fair correlation with the Simple Shoulder Test (r = 0.67) and the American Shoulder and Elbow Surgeons shoulder (r = 0.63) assessment instruments. Although these item banks offer the potential for use in athletic training practice and research, the measurement properties of the PROMIS physical function item bank must first be established for individuals who represent high levels of physical ability. Therefore, the purpose of our study was 2-fold. First, we examined whether the PROMIS physical function item bank, version 1.0, in computer-adaptive form could accurately distinguish collegiate-level athletes based on health status. We hypothesized that the mean PROMIS scores for healthy (HC), injured and participating (IP), and injured and not participating (INP) collegiate athletes would differ, making the test a useful tool for monitoring functional improvements over the course of rehabilitation. Second, we compared the mean PROMIS scores of HC athletes with those of healthy intramural athletes to assess the ability of this instrument to measure individuals with high levels of physical ability. For this secondary purpose, we assumed that collegiate athletes inherently possess a higher degree of physical function than individuals who participate solely in intramural activities, as collegiate athletes are exposed to daily training regimens and structured sport-conditioning programs. Under this assumption, we hypothesized that the mean PROMIS scores for collegiate athletes would be different from those of the intramural group, providing evidence that the current PROMIS item bank does not suffer from ceiling effects.

METHODS

Statistical Analysis

We analyzed the mean PROMIS physical function scores for the HC, IP, INP, and HR groups using SPSS (version 12; SPSS Inc, Chicago, IL). First, a 1-way analysis of variance was calculated to compare scores among the 3 collegiate groups: HC, IP, and INP. An independent-samples t test was used to investigate differences between the healthy groups: HC and HR.

To further assess item specificity after data collection, we divided the HC participants into 2 subgroups (distance-running athletes [DRAs] and sprinting, agility, and power athletes [SAPAs]; Table). Using an independent-samples t test, we compared the mean PROMIS physical function scores between the 2 subgroups of the HC participants: DRA and SAPA.

Table Categorization of Distance-Running Versus Sprinting, Agility, and Power Sports

View Fullscreen

RESULTS

We found differences among the HC, IP, and INP groups (F_2,149 = 67.42, P < .001; Figure A). A Tukey honestly significant difference post hoc analysis showed that the HC group (mean = 61.7 ± 6.0) had a higher PROMIS score than either the IP (mean = 54.9 ± 7.5) or the INP (mean = 44.1 ± 8.2) group, and the IP group had a higher score than the INP group. We observed no difference in physical function scores between the HC and HR groups (mean difference = 1.90, t₁₃₃ = 1.68, P = .10; Figure B). Finally, we noted no difference between the mean physical function scores of the DRA and SAPA subgroups (mean difference = 1.65, t₆₈ = 1.18, P = .25; Figure C).

View Figure

• Download as Powerpoint
• Download Figure

DISCUSSION

Incorporating item response analysis into routine PRO assessment allows practitioners to estimate a latent trait (in this case, physical function) independently from the specific characteristics of the population.²¹ Before this can be accomplished, item banks must represent a wide range of ability levels.⁵ According to our results, the PROMIS physical function computer-adaptive test appears to contain question items that can accurately monitor elite-level athletes at various stages of musculoskeletal injury recovery.

We found differences among the mean PROMIS physical function scores of each collegiate athlete group; the lowest mean score belonged to the INP group and the highest mean score belonged to the HC group. After we compared these 3 groups, perhaps the most interesting finding was the difference in mean PROMIS scores (6.83) between the IP and HC athletes. According to our inclusion criteria, IP athletes had musculoskeletal injuries that required medical intervention but lacked the signs and symptoms that would otherwise require them to be withheld from competition. These individuals are frequently treated by ATs because of the diminished functional performance that typically accompanies such injuries. Therefore, an important step in improving PRO assessment for active patient populations is identifying an outcome instrument that can accurately differentiate healthy athletes from athletes participating despite minor functional limitations due to injury.

Upon closer inspection, the question “Does your health now limit you in doing strenuous activities such as backpacking, skiing, playing tennis, bicycling or jogging?” appears to be specific and difficult enough to distinguish HC from IP collegiate athletes. The computer-adaptive test administered this question to 124 of the 150 collegiate athletes, including every HC and IP participant (n = 120) and only 4 INP participants. The HC athletes who were currently participating in sporting events without any discomfort answered this question using the response option not at all, whereas the majority of IP athletes responded with very little. Ultimately, the slight variance in response options for this question led to 2 different assessments. Because of the principles of item response theory, any reported difficulties with these functional tasks could be attributed to a certain level of functional limitation.¹³ The possible range of physical function scores for these participants began to narrow, and the number of questions needed to accurately place the participant along the latent trait continuum decreased dramatically.¹⁰ The computer continued to administer more questions to the HC participants because the range of possible physical function was so broad. As a result, before receiving a score, the HC participants answered an average of 10 questions, whereas the IP participants answered an average of only 4 questions.

A secondary goal of our study was to determine if the PROMIS physical function item bank was limited by a potential ceiling effect in assessments of athletic training clientele. In other words, did the physical function scores appear to plateau when this outcomes instrument was used to measure physical function in individuals with high levels of physical ability? To test this, we compared the mean PROMIS physical function scores of the HC and HR groups. According to our analysis, the scores of these groups were not different. This result seems surprising given that collegiate scouts and coaches tend to recruit the most athletic individuals from the general population. In addition, collegiate athletes participate in regular team training sessions designed to increase physical ability and improve athletic performance so as to remain competitive in organized collegiate sports. Although intramural athletes may work out regularly, they are not often exposed to the same structured conditioning programs or competitive athletic environments that collegiate athletes experience.

Several explanations are possible for the similarity in HC and HR scores. First, our assumption that collegiate athletes possessed higher levels of physical ability could have been wrong. Intramural and collegiate-level athletes may, in fact, not be different in functional ability. A second explanation is that the PROMIS physical function item bank has a ceiling effect when participants possess an inherently high level of functional ability. Yet a third explanation could be poor item specificity rather than the presence of a ceiling effect. For instance, the most difficult items included in the 124-item bank are questions such as “Are you able to run 2 miles at a fast pace?”, “Are you able to run 5 miles?”, and “Are you able to run 10 miles?” These questions do not necessarily pertain to the physical skills frequently performed by collegiate athletes. The physical attributes that set collegiate athletes apart from recreationally active adults are not always related to running long distances. Instead, some collegiate athletes may be more likely to pursue conditioning programs designed to improve power and perform exercises such as the squat, bench press, and dead lift than distance running. Therefore, many of the collegiate athletes in our healthy participant pool may have had the same difficulty running long distances as adults in the general public.

Before collecting data, we were aware that the most difficult question items included in the PROMIS item bank focused on long-distance running activities. However, for our initial analysis, we grouped HC athletes regardless of their sport. To further assess item specificity, we performed a separate analysis in which the HC athletes were divided into 2 subgroups depending on the physical demands of their sport. According to this analysis, the mean PROMIS physical function scores for DRAs and SAPAs were similar despite their having different training regimens. Even though SAPAs were rarely exposed to long-distance running, we cannot necessarily conclude that these athletes were responsible for lowering the average physical function score for the entire HC group and making them look similar to the HR group. In fact, scores for the healthy cross-country runners who trained specifically for long-distance running events were among the lowest scores in the HC group. Because cross-country runners often run for extended periods of time, they may have a better understanding of how difficult running 5 to 10 miles actually is. In other words, even though these items may be difficult, only certain types of athletes understand the full magnitude of these tasks.

The original goal in developing the PROMIS item banks was to create a set of questions that could measure PROs across common medical conditions²² and not to distinguish healthy athletes based on sport type. However, to produce truly accurate scores across athletes in all sports, our results suggest that high levels of physical function should not be attributed solely to an increase in running distance. Instead, the PROMIS physical function item bank, version 1.0, should include items that are specific to all sport skills so that athletes accurately comprehend the difficulty of each task.

CONCLUSIONS

The PROMIS physical function item bank, version 1.0, appears to be a useful instrument for assessing physical function in athletes recovering from musculoskeletal injuries. The 124-item bank accurately differentiated physical function of the HC, IP, and INP athletes. Sports medicine clinicians looking to assess the effectiveness of a treatment in the early stages of a rehabilitation protocol will find the PROMIS physical function item bank beneficial. In particular, ATs will appreciate the added benefits of diminished time constraints that accompany computer-adaptive instruments. However, we found that relying on descriptions of long-distance running tasks to measure elite levels of physical function was insufficient. As collegiate athletes rehabilitate and begin practicing functional tasks specific to their sport, the PROMIS item bank does not appear to contain questions that are sufficiently specific to provide an accurate assessment across all sports. Researchers should make every effort to collaborate with PROMIS investigators in the future so that the most accurate item banks can be developed for sport-specific clienteles.