Acute Stimulant Ingestion and Neurocognitive Performance in Healthy Participants

The management of mild traumatic brain injury, typically referred to in the United States as a concussion, has become an area of great concern and controversy in the athletic setting. Numerous states have adopted or are in the process of adopting legislation that mandates evaluation procedures and return-to-participation criteria for interscholastic athletics, and many sport governing bodies have adopted similar policies. Concussions are typically diagnosed after several clinical domains are assessed, including self-reported symptoms, physical signs, and cognitive functioning. Current strategies for injury management suggest that abnormalities in any 1 or more of these domains should exclude an athlete from training and competition.

Many clinicians use neurocognitive tests, such as Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT; ImPACT Applications, Inc, Pittsburgh, PA), as assessment and management tools. These tests are usually administered before the sport season to obtain baseline data. If an athlete is concussed, serial evaluations are conducted postinjury to determine when neurocognitive deficits and clinical symptoms are no longer present. Researchers¹ have recommended that neurocognitive performance must revert to baseline or better before the patient returns to participation to reduce the possibility of a more serious, cumulative injury during the vulnerable recovery period.

The restrictive nature of many current management plans has resulted in a longer recovery period and a delay in return to participation compared with older management plans and has created controversy. Authors^2–4 of several peer-reviewed publications have reported that athletes ranging from the high school to the professional level have stated that they have hidden or would hide the symptoms of a concussion from a health care professional or a coach primarily because they feared missing participation time or status. Similarly, anecdotal concerns have been raised about athletes trying to cheat the assessment and return to participation sooner than they should.

Various stimulants, including caffeine, have been shown to improve working memory, visual information processing, and reaction time (RT), which are similar to the skills used during ImPACT.^5–7 Thus, stimulant use could possibly improve performance during neurocognitive concussion testing. However, no researchers have examined the effects of stimulant use on ImPACT or any other type of computerized neurocognitive testing. Therefore, the purpose of my study was to determine if acute stimulant ingestion would improve performance during ImPACT. I hypothesized that memory, processing speed (PS), and RT would improve after stimulant ingestion.

METHODS

Instruments

The Web-based version of ImPACT was used to assess neurocognitive function. This computerized test battery consists of 6 modules that evaluate attention, verbal recognition memory, visual working memory, visual PS, RT, numerical sequencing ability, and learning. The modules are presented in a near-infinite number of alternate forms to minimize practice effects and yield 5 separate composite scores that are used to assess neurocognitive function (Table 1). The composite scores are constructed to better isolate test performance within the cognitive domains of verbal memory (M_Verb), visual memory (M_vis) PS, RT, and impulse control (IC). In addition, a cognitive-efficiency index (CEI) is provided, which measures both speed and accuracy on the symbol match test. The reliability⁸ and validity⁹ of the composite scores and the sensitivity and specificity¹⁰ of the test battery for identifying neurocognitive changes have been reported.

Table 1. Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT)^a Neurocognitive Composites

View Fullscreen

RESULTS

A difference was observed when comparing the RT composite, as participants scored better during the S condition than during the P and C conditions (F_2,22 = 4.31, P = .03; Table 2). No difference was observed when comparing the P and C conditions. I also noted a difference for the CEI, as participants scored higher during the S condition than during the P and C conditions (F_2,22 = 4.07, P = .03; Table 2). Again, no difference was evident when comparing the P and C conditions. However, I found no differences in M_Verb (F_2,22 = 0.938, P = .41), M_vis (F_2,22 = 0.213, P = .81), PS (F_2,22 = 0.415, P = .67), or IC (F_2,22 = 0.602, P = .56) when comparing treatment conditions. Post hoc analysis of observed power revealed values of .19, .08, .11, .69, .14, and .66 for the M_Verb, M_vis, PS, RT, IC, and CIE, respectively.

Table 2. Immediate Post-Concussion Assessment and Cognitive Testing (ImPACT) Neurocognitive Composite Scores During Control, Placebo, and Supplement Conditions

View Fullscreen

I noted differences in M_Verb (F_3,33 = 3.52, P = .03) and the CEI (F_3,33 = 6.00, P = .002) when comparing the test sessions, including baseline, by the order of testing. An improvement was demonstrated when comparing serial test sessions to the baseline session for both variables, whereas no differences were seen when comparing any of the serial test sessions. No differences were observed for M_vis (F_3,33 = 2.62, P = .07), PS (F_3,33 = 2.78, P = .056), RT (F_2,22 = 1.32, P = .28), or IC (F_3,33 = 2.74, P = .059). However, trends suggested improvements during serial testing for M_vis, PS, and IC compared with baseline. In addition, no differences in the hours of sleep the night before testing (F_2,22 = 0.347, P = .71) were shown when comparing the S (7.3 ± 1.2 hours), P (7.0 ± 2.1 hours), and C (7.0 ± 1.5 hours) conditions, and no differences were evident when comparing the carbohydrate (F_2,22 = 1.32, P = .29), fat (F_2,22 = 1.40, P = .27), or protein (F_2,22 = 0.251, P = .78) content or the total caloric intake (F_2,22 = 1.25, P = .31) before testing during each condition. Similarly, I observed no differences when comparing HR (F_2,22 = 0.767, P = .48), SBP (F_2,22 = 0.106, P = .90), or DBP (F_2,22 = 1.43, P = .26) before or after ingestion during the S, P, and C conditions.

DISCUSSION

The management of sport-related concussion can be one of the most challenging endeavors for the health care professional, as the short-term and potential long-term sequelae present important concerns. Therefore, more than 20 different management guidelines have been published, providing recommendations for concussion diagnosis, management, and return-to-participation decisions. Whereas many of them use grading scales based solely on a loss of consciousness and posttraumatic amnesia, the current Consensus Statement on Concussion in Sport¹¹ recommended that injury-grading scales should be abandoned in favor of assessing a range of domains to determine injury severity and to guide individual return-to-participation decisions. This multifaceted approach includes neurocognitive testing that, when used in conjunction with self-reported symptoms, can increase diagnostic accuracy and better predict recovery.^12,13 Thus, more recent concussion-management protocols have recommended that athletes undergo neurocognitive testing both before and serially after sustaining concussions, thereby allowing the clinician to compare baseline with postinjury data. However, comparing the baseline with postinjury testing accurately identifies cognitive changes only if both the baseline and postinjury tests are valid. Hence, clinicians need to identify any potential threats to this validity. My results suggested that acute ingestion of a nutritional supplement containing caffeine and DMAA improved neurocognitive performance and therefore threatened ImPACT validity. Although I noted no differences in M_Verb, M_vis, PS, and IC, I observed improvements in RT and the CEI. The RT and CEI improved after stimulant ingestion when compared with both the C and P conditions, whereas no differences were observed when comparing the P and C conditions. I chose Jacked 3D for this investigation because of anecdotal reports and personal observations of its popularity. This dietary supplement contains several ingredients, including the stimulants caffeine and DMAA. Researchers^14–16 have reported the effects of this combination on metabolism and physical performance but not on cognitive performance. Unfortunately, determining which ingredient was primarily responsible for the cognitive changes was not within the scope of my study, and no investigators have investigated the effect of DMAA alone on these measures. In contrast, improved cognitive function after caffeine ingestion has been reported.^5–7 My results were similar to those reported by authors using other measures of cognitive performance, but I am the first, to my knowledge, to use computerized neurocognitive tests to assess the effects of caffeine or any other stimulant.

Impaired RT is one of the most common cognitive sequelae of concussion and represents one of the most sensitive indices of cognitive change, as deficits in RT have been observed after concussion without changes in learning or memory.^17,18 This might explain why I observed differences in the RT composite without changes in the M_Verb, M_vis, PS, and IC composites and with improvements in the CEI, as the combination of response accuracy and speed determines this measure. The CEI is derived from the symbol match test, which also is used to determine RT. Thus, a correlation between differences in RT and in the CEI would be expected. These improvements are clinically important, as impaired RT has been observed in patients even after they are asymptomatic.^19–22 Thus, RT should be considered an important component of a concussion-assessment battery.

Whereas I observed differences in RT and the CEI, the question of clinical importance must be considered. Nine of the 12 participants had faster RTs after stimulant ingestion than in the C condition, with an average improvement of 0.04 seconds. However, no participant exceeded the reliable-change index score previously reported (80% confidence interval = 0.06 seconds).⁸ The reliable-change index allows a clinician to establish empirically derived cutoff scores that can be used for evaluating meaningful differences independent of psychometric factors, such as practice effects and other sources of variance. At this time, no universally accepted cutoff scores exist for return-to-participation decisions associated with ImPACT. Thus, the importance of my results must be determined by the clinician's own clinical judgment. Although the observed improvements were not equivalent to some of the deficits reported acutely after concussion,^13,20 they eventually could mask deficits during serial testing. This could potentially influence return-to-participation progression and may be of even greater concern because a rapid RT is necessary to avoid injury. Again, the clinician must decide if the small changes represent clinically important differences. As noted, I assessed healthy individuals rather than those diagnosed with concussions. Therefore, I do not know if an injured population would experience the same response. The observed improvements were consistent with reported observations of improved RT in healthy individuals after caffeine ingestion, with doses ranging from 60 mg (a typical cup of coffee) to 250 mg.^5–7 Unfortunately, the dose of caffeine in my study was unknown because supplement manufacturers are not required to disclose this information on the product labels.²³ The same is true for the DMAA dose, as the ingredients were listed only as a proprietary blend. Not all researchers have reported RT improvements after caffeine ingestion.²⁴ Similarly, not every participant in my study experienced improvements. The specific dose used, as well as the level of habitual caffeine use, certainly can influence the response.^7,25 Peak plasma-caffeine concentration is reached between 15 and 120 minutes after oral ingestion, with a half-life of 2.5 to 4.5 hours, and I administered the supplement 30 minutes before testing. All participants received the same dose, as I chose not to administer it relative to body mass. Thus, my participants likely had varying rates of absorption, distribution, and elimination because they had a wide range of body masses (54–98 kg). However, I believe this approach is more clinically relevant, as the directions for use on most supplement labels include an absolute dose and do not account for body mass. I also did not control for the level of habitual caffeine use but instructed participants to report their use. Two participants were caffeine naïve, whereas the remaining participants' habitual use ranged from approximately 60 to 160 mg/d. From these data alone, I could not determine if a correlation existed between levels of habitual use and changes in RT and the CEI. Other investigators^7,25 have shown that varying levels of habitual caffeine use can affect cognitive performance, so I cannot rule out this potential influence on my results.

The participants' levels of arousal at the time of testing can also influence neurocognitive performance, so I also attempted to control for these extraneous variables. To account for circadian rhythm, participants were tested on the same day of the week and at the same time of day for each session. I instructed participants to go to bed at the same time on the nights before testing. I recorded and analyzed the hours of sleep that participants had each night before testing and observed no differences. Finally, to control for possible dietary influences on arousal, participants were instructed to record their diet on the day of the first test session and to follow that diet on each subsequent test day. From my analysis of the dietary record, I determined that participants were compliant with their carbohydrate, fat, protein, and total caloric intakes.

Comparing a patient's postinjury performance with his or her own baseline performance is preferable to reduce extraneous variability not attributable to the concussion, but repeated administrations of the test battery might underestimate concussion severity because of practice or learning effects.^26,27 This was also possible in my study. Researchers¹⁷ have suggested that the absence of this improvement, as opposed to deficits, might suggest an initial decrement in cognitive function. The use of computerized testing promoted a more accurate measurement of cognitive processes, such as PS and RT. Computerized assessments of RT are accurate to 1/100 of a second, whereas traditional testing allows for accuracy only to 1 to 2 seconds. Iverson et al⁸ suggested that this increased accuracy improved the validity of test results in detecting subtle changes in neurocognitive processes. The use of the computer also allowed test stimuli to be randomized, which should have improved reliability across multiple administration periods, minimizing the practice effects that naturally occur with multiple exposures. The ImPACT modules I used were presented in a near-infinite number of alternate forms to minimize practice effects during repeated testing. In addition, to reduce the influence of practice, I used a randomized and counterbalanced crossover design by having participants complete the test battery during an initial familiarization session and by separating the testing sessions by 1 week. Each session after the initial session was treated as postinjury serial testing; thus, the specific test battery was never repeated. I also analyzed the data by the order of test session and observed improvements when comparing serial test sessions with the baseline test session. However, no differences were evident when comparing the weeks of serial testing. Thus, I am confident that the improvements observed in RT and the CIE were due to the supplement and not to learning or practice effects.

As mentioned, neurocognitive tests are used with clinical examination and symptom reporting when diagnosing and managing concussion. Thus, my findings are of clinical importance, as they raise questions regarding test validity and the influence on return-to-participation progression. This factor is of particular concern if the validity of symptom reporting also can be questioned. Athletes possibly would be motivated to minimize symptoms to permit an early return to participation. Several authors^2–5 of peer-reviewed publications reported that athletes ranging from the high school to the professional level have stated that they have hidden or would hide the symptoms of concussion from a health care professional or a coach. The devotion to their sports, teams, and future athletic careers is a powerful motivator for high school and college athletes. Similarly, athletes' fear of losing their positions on the team also may tempt them to deny or underreport postconcussive symptoms. When such underreporting is suspected, findings on objective neurocognitive measures of memory, PS, and RT could provide valuable information for determining an athlete's readiness for return to competition. For example, neurocognitive impairments have been observed in concussed athletes who were asymptomatic at the time.^14,19–21 This finding suggested that full recovery had not occurred, even in the absence of reported symptoms, and that without neurocognitive testing, a premature return to participation might have occurred. Premature return to participation may have serious consequences if the athlete is still in a vulnerable state. However, athletes who would hide symptoms might also be motivated to try to cheat neurocognitive tests for an earlier return to participation. My results suggested that stimulant use can enhance cognitive performance and provide a method for cheating the test. This is alarming, as premature return to participation presents an important concern after concussion. Guskiewicz et al²⁸ found that 1 in 15 college football players with concussions may have sustained additional concussions in the same playing season and that these reinjuries typically occurred in a short window of time (7–10 days) after the first concussion. Their observations may be a greater concern in adolescent athletes, who have demonstrated greater duration and severity of neurocognitive dysfunction than concussed college athletes.²⁹ Furthermore, a premature return to participation has been associated with delayed recovery time and potentially catastrophic consequences, such as diffuse cerebral swelling or second-impact syndrome.²⁹ Whereas disagreement exists over the incidence of second-impact syndrome, many researchers³⁰ agree that the syndrome is rare. However, second-impact syndrome must still be taken seriously, as the consequences could be grave.

A concern with using any supplement is the potential for side effects. Several minor and serious side effects are associated with using any stimulant, including caffeine. No participant reported any adverse effects after the P or C conditions. However, 1 participant reported jitters and anxiety after the S condition, which are not uncommon after stimulant ingestion. No changes in HR, SBP, or DBP were demonstrated during any of the conditions. Similar observations have been made after Jacked 3D ingestion.³¹ The cardiovascular response after caffeine ingestion varies, with some reporting changes and others not reporting them.^31,32 Similar to the neurocognitive response, varying levels of habitual caffeine use can affect the cardiovascular response.³² As previously stated, the exact dose of DMAA and caffeine contained in my supplement was unknown; however, based on the experience of the participants, I assumed it was within a safe range.

An obvious limitation to this study was that I assessed healthy rather than concussed participants. Thus, I do not know if an injured population would experience the same response. I also combined men and women and was limited to a small sample size, which further restricted my ability to generalize the results to a concussed population. The US Food and Drug Administration recently removed products containing DMAA from the marketplace because of questions about its natural origin and safety. However, some manufacturers continue to distribute products adulterated with DMAA, and caffeine remains legally available for widespread sale. Thus, researchers should assess ImPACT using caffeine alone and in doses commonly found in popular energy drinks. Investigators should also assess the effects of prescription stimulants, such as Adderall (Shire US Inc, Wayne, PA) and Ritalin (Novartis Pharmaceuticals Corporation, East Hanover, NJ), which recently have become popular as potential performance enhancers. The purpose of my investigation was not to inform athletes of this potential cognitive performance enhancer but to make clinicians aware of this potential.

CONCLUSIONS

Neurocognitive testing is 1 element of a multifaceted approach to concussion assessment and management. When these tests are used, comparing postinjury data with baseline data accurately identifies cognitive changes only if both the baseline and postinjury tests are valid. Care should be taken to provide each athlete with an environment that is designed to maximize test performance and be reproduced easily in the postinjury setting. Therefore, clinicians should standardize and control test conditions to increase the validity of ImPACT during baseline and postinjury testing. My results suggested that this should include controlling for caffeine and other stimulant use before and during testing. Many athletes might habitually ingest foods or drinks containing caffeine or other stimulants without knowing the potential influence on neurocognitive tests and without the intent of influencing their performance, whereas others might do so purposely. Controlling for overall diet before testing may also be beneficial, because other dietary influences exist and because the results of these objective tests are used with self-reported symptoms and the clinical examination when determining return-to-participation progression after head trauma. Our primary concern is the overall health and safety of our athletes.