Electronic Patient-Reported Outcome Validation: Disablement in the Physically Active Scale

Patient-reported outcomes measures (PROMs) provide information on a patient's perception of his or her health status, disability, and treatment effects.¹ Assessing information from the patient's perspective helps clinicians enhance their understanding of individual patient needs and make more precise treatment decisions.¹ Health care professionals have commonly used paper-and-pencil forms to collect PROMs, but as technology advances, they are more interested in delivering PROMs electronically to make the process more efficient. The shift to electronically administered PROMs necessitates establishing that electronic PROMs are equivalent to the original paper-and-pencil versions to ensure accuracy and dependability of the measures in different forms.²

Whereas PROMs are used frequently throughout the health care professions, their application in athletic training has been sporadic.¹ Many reasons exist for the lack of PROM collection in athletic training, and a common obstacle is time.¹ Clinicians have reported that scoring and interpreting PROMs might occupy valuable time that could be devoted to patient care. A perceived increase in the demand for athletic trainers to collect, interpret, file, and maintain this additional information has been cited as a primary concern.¹

Using electronic methods (eg, computers, tablets) may resolve some of the clinician-perceived barriers to PROMs collection and could enhance communication between the patient and clinician, thereby improving patient assessment and care.¹ Electronic collection also requires less physical space because data are directly entered into a small storage device or on a remote cloud server, reducing the need for paper and storage.² In addition, when patients complete PROMs electronically, they answer more quickly, and their answers are recorded directly in an electronic database, eliminating manual entry time and reducing data-entry errors.³ Furthermore, electronic instruments can be programmed to require a response before a patient proceeds to the next question, which can reduce missing data.³ After the information is electronically collected, scores and results can be calculated through computerized automation, further reducing the time demand on the clinician.⁴

However, before electronic versions of PROMs can be used with confidence, the electronic version must be assessed to determine its validity compared with the original. This step is necessary to determine whether the electronic and paper versions of PROM instruments measure the underlying constructs in the same fashion. Without this step, the validity of PROMs completed electronically cannot be guaranteed.^3–5

We selected the Disablement in the Physically Active (DPA) scale as an instrument to validate in electronic format, as it is commonly used in treating physically active participants with musculoskeletal injuries and has immediate clinical applicability to athletic trainers.^5,6 The instrument has been recommended as a valid and responsive tool for patient care and research. However, an electronic version has not been evaluated for accuracy. Therefore, the purpose of our study was to advance the use of the DPA scale by determining the validity of an electronic version of the scale compared with its paper version.

METHODS

Students in physical activity classes and intercollegiate athletes at East Los Angeles College were invited to participate. Physically active male and female volunteers aged 18 to 55 years were recruited. Physically active individuals were defined as those who engaged in activity requiring physical skill that incorporated power, strength, speed, endurance, agility, flexibility, or range of motion at least 3 times per week.⁶ Both uninjured and injured volunteers were included. Injury status was categorized as healthy or having an acute, subacute, or persistent injury as reported by the participant, according to the definitions outlined in the Table. We excluded volunteers who did not meet the physical activity level or age requirement or reported chronic pain, as these can result in periods of incapacity, inconsistent activity, and unpredictable patterns.⁷ The Institutional Review Board of the University of Idaho determined the study to be exempt.

Table Participant Demographics Summarized by Sex and Injury Level

View Fullscreen

RESULTS

A total of 117 participants (38 women, 79 men; age = 21.6 ± 5.9 years) met the inclusion criteria and completed both assessment modes as outlined in Figure 1. Most participants reported being healthy (n = 99, 85%); 18 (15%) reported having an acute (n = 11), subacute (n = 2), or persistent (n = 5) injury. Participant demographics, including sex and injury status, are summarized in the Table.

View Figure

• Download as Powerpoint
• Download Figure

A strong bivariate correlation between total DPA scale scores was found between the testing modes (r = 0.86, R² = 0.74, P < .001). The canonical correlation indicated 3 relationship patterns among scale subdimensions across testing modes. The first correlation was R = 0.90 (R² = 0.81, P < .001), the second was R = 0.74 (R² = 0.55, P < .001), and the third was R = 0.69 (R² = 0.48, P < .001). A strong relationship (r = 0.90, R² = 0.81, P < .001) between the testing modes was also found when assessing a first-order latent variable of disablement (Figure 2).

View Figure

• Download as Powerpoint
• Download Figure

DISCUSSION

We compared paper-and-pencil and electronic administration of the DPA scale and observed that scores on these versions were strongly correlated. Bivariate analysis of summative scores, using a Pearson product moment correlation, indicated a high correlation between participant scores using both administration methods. The multivariate-manifest and latent-variable approaches, which are more rigorous statistical tests, revealed stronger relationships between testing administration modes. The strong correlation value of the latent-variable analysis suggested that participants provided equivalent responses across items when completing the paper and electronic versions of the scale.

The canonical correlation analysis of manifest scores also supported the latent-variable analysis across multiple relationship patterns. Examination of the loadings for the first relationship pattern indicated that participants reported minimal physical dysfunction (eg, low scores on functional limitations, impairments, and disability scores) with a high QOL across both testing modes. The second relationship pattern suggested a group of respondents who reported minimal physical dysfunction but impaired QOL across both testing modes. The third relationship pattern indicated low scores in the impairment dimension but moderate perceived dysfunction across the functional limitations, disability, and QOL dimensions in both testing modes. The correlation results provided support for electronic completion of the DPA scale as a valid administration method.

Patient-reported outcome measures are becoming more commonly used in athletic training and across health care professions, but barriers persist.¹ One solution to these barriers is electronic PROMs, which simplify data collection, can provide reliable electronic data in various areas of health care,^3,4 and may enhance patient-assessment efficacy and treatment direction.¹ In a recent meta-analysis³ of the equivalence of 278 PROMs in paper and electronic forms, the average weighted correlation was 0.90, with 94% of the correlations ≥0.75. The results of our study are in line with those findings,³ but one cannot assume that an electronic PROM is equivalent to the paper counterpart until it is tested. Equivalency studies must be conducted before clinicians can confidently implement electronic PROMs.

Further research is needed to establish the validity of the DPA scale. For example, researchers should conduct a cross-validation study among a larger, more diverse sample to allow for confirmation (ie, confirmatory factor analysis) of the DPA scale. Furthermore, multigroup (eg, stability between subgroups) and longitudinal invariance (ie, stability of measurement variables over time within the same population) testing must be completed on the DPA scale. This also should be applied to other PROMs used in health care, as many need to be tested among larger, more heterogeneous populations to establish initial construct validity while also being tested in subsamples of the population to ensure that measurement bias is not present in different groups.

CONCLUSIONS

Electronic completion of the DPA scale resulted in responses equivalent to those on the paper-and-pencil version. Athletic trainers may be more inclined to use electronic PROMs because they are more easily and accurately completed by patients, save time, decrease the amount of paper waste, and increase record-keeping efficiency.⁸ To realize the benefits of electronic PROMs in clinical practice, more equivalency studies need to be conducted.