Mild traumatic brain injury (TBI) or concussion is common in many sports. Today, neuropsychological evaluation is recommended in the monitoring of a concussion and in return-to-play considerations. To investigate the sensitivity of neuropsychological assessment, we tested amateur boxers post bout and compared with controls. Further the relationship between neuropsychological test results and brain injury biomarkers in the cerebrospinal fluid (CSF) were investigated.
Thirty amateur boxers on high elite level with a minimum of 45 bouts and 25 non-boxing matched controls were included. Memory tests (Rey Osterrieth Complex Figure, Listening Span, Digit Span, Controlled Word Association Test, and computerized testing of episodic memory), tests of processing speed and executive functions (Trail Making, Reaction Time, and Finger Tapping) were performed and related to previously published CSF biomarker results for the axonal injury marker neurofilament light (NFL).
The neurological assessment showed no significant differences between boxers and controls, although elevated CSF NFL, as a sign of axonal injury, was detected in about 80% of the boxers 1–6 days post bout. The investigation of the relationship between neuropsychological evaluation and CSF NFL concentrations revealed that boxers with persisting NFL concentration elevation after at least 14 days resting time post bout, had a significantly poorer performance on Trail Making A (p = 0.041) and Simple Reaction Time (p = 0.042) compared to other boxers.
This is the first study showing traumatic axonal brain injury can be present without measureable cognitive impairment. The repetitive, subconcussive head trauma in amateur boxing causes axonal injury that can be detected with analysis of CSF NFL, but is not sufficient to produce impairment in memory tests, tests of processing speed, or executive functions. The association of prolonged CSF NFL increase in boxers with impairment of processing speed is an interesting observation, which needs to be verified in larger studies.
Citation: Neselius S, Brisby H, Marcusson J, Zetterberg H, Blennow K, Karlsson T (2014) Neurological Assessment and Its Relationship to CSF Biomarkers in Amateur Boxers. PLoS ONE 9(6): e99870. https://doi.org/10.1371/journal.pone.0099870
Editor: Sonia Brucki, University Of São Paulo, Brazil
Received: February 4, 2014; Accepted: May 20, 2014; Published: June 18, 2014
Copyright: © 2014 Neselius et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by ALF Grants, County Council of Västra Götaland, Sweden; Gothenburg Medical Society, Sweden; and ALF Grants, County Council of Östergötland. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Mild traumatic brain injury (TBI) or concussion is defined as a complex pathophysiologic process affecting the brain, induced by traumatic biomechanical forces with or without loss of consciousness . They represent 80–90% of all traumatic brain injuries  and account for between 5.8–8.9% of all athlete injuries among United States high school and collegiate athletes . It is also worth pointing out that most athletes experiencing a concussion, up to 70%, do not seek medical attention . Since concussion is now recognized as an increasing and serious health problem in many sports, it has gained increased attention among medical professionals, scientists, sport organizations as well as athletes.
A TBI causes axonal and glial damage, which disturbs the cerebral physiology, and makes the brain more vulnerable for additional concussions. The primary neuropathology of mild TBI is a diffuse axonal injury (DAI) , . It is caused by shearing of fragile axons, due to acceleration and deceleration forces during the trauma . The DAI inflicted by a mild TBI, causes predominantly cell dysfunction, rather than cell death . A boxer can receive a mild TBI due to a knockout punch. The knockout frequency in modern amateur boxing is very low, in average 0.7% , however, it is known that also the repetitive subconcussive trauma in boxing causes axonal injury –.
Neuropsychological Evaluation of Mild Traumatic Brain Injury
Neuropsychological evaluation has been advocated as the most sensitive tool for early detection of central nervous system pathology and numerous studies have shown that both traditional and computerized neuropsychological tests (emphasizing mental processing efficiency and speed) are sensitive for the acute cognitive impairment after a mild TBI up to 10 days post injury –. Also in studies where patients with mild TBI have been followed for longer time periods have concluded that neuropsychological assessment in mild TBI are of clinical value and contributes with significant information in the evaluation, in combination with symptom and physical assessment , . In a meta-analysis of neuropsychological outcome after a single mild TBI, the largest impairment was seen in verbal, visual and working memory (attention, concentration, processing speed) up to 30 days post trauma . However, the largest effect was observed during the first seven days and no neuropsychological changes were detectable after three months . In a study on 698 jockeys decrements in high-level executive/attentional functioning after multiple concussions, persisting more than three months post trauma, were seen . Younger athletes were here found to be more vulnerable to concussions than adults.
For amateur boxing there is a recently published study showing cognitive impairment, after self-reported concussions (knockouts), when compared to controls . Here, cognitive recovery occurred within a week. In this study the computerized test Automated Neuropsychological Assessment Metrics (ANAM)  was administered and the study objects in the boxer group were tested at baseline and post injury and thereafter compared to a group of baseline tested boxers without concussion. However, to our knowledge, no neuropsychological assessment has been able to show any pathology caused by the repetitive subconcussive trauma in amateur boxing –, indicating that there are no cognitive tests that are sensitive enough to detect small axonal injuries.
Neuropsychological Tests used in the Diagnosis and Monitoring of TBI
The Rey-Osterrieth Complex Figure (ROCF) Test is commonly used to test episodic memory and visuospatial skills . It correlates with the Trail Making test and Wechsler Adult Intelligence Scale – Revised (WAIS-R) . A study comparing subjects with traumatic brain injury (TBI) and patients with Alzheimer’s disease (AD) showed that the TBI subjects did not have any dysfunctions in copying the figure, only with recall, but the AD subjects had dysfunctions both with copying and recall . There are also computerized possibilities, for example testing made by a model from Tulving , to evaluate episodic memory.
Language and semantic memory.
Vocabulary intervention is part of the WAIS-R and related to level of education and is critical for studies where educational background may interfere with neuropsychological results .
Controlled Oral Word Association Test (COWAT) is a part of the Multilingual Aphasia Examination and provides a measure of word retrieval and word fluency , . A study on 96 athletes suffering from a sport-related concussion did not reveal significant difference with controls , but in mild cognitive impairment (preclinical stage of AD), impairment was seen compared to controls .
Listening span Test has been used extensively in our laboratories  and measures complex, executive aspects of working memory that are related to short-term memory capacity , . It is impaired in preclinical stages of AD  but seems not to be affected by sport-related concussions . Digit Span Test is part of the Wechsler Adult Intelligence Scale – Revised (WAIS-R) and addresses short-term auditory memory that is part of the working memory . It can be used to monitor less effortful attention skills, in contrast to the Listening Span task, that involves effortful working memory and attention skills. Typically, performance on the Digit Span task is relatively insensitive to effects of mild TBI . However, it ascertains that participants master necessary attention skills in order to allow meaningful interpretation of other neuropsychological data.
Processing speed and executive functions.
Trail Making Test is used for assessing processing speed, attention and executive functioning . There is ample evidence that it can detect brain damage and predict long-term outcomes after traumatic brain injury –. Reaction time Test is sensitive to the effects of mild TBI .
Evaluation of Biomarkers in Cerebrospinal Fluid and Blood in this Cohort of Boxers
We have previously shown that amateur boxing causes elevation of brain injury biomarkers in blood and CSF –. CSF Neurofilament Light Protein (NFL), a marker of axonal injury, was shown to be the most sensitive biomarker and to correlate with the amount of injury . NFL levels were elevated in 77% of the boxers 1–6 days after a bout as a sign of mild TBI, when compared to matched controls. After a rest period of at least 14 days, the levels were still elevated in 46% of the boxers .
Aim of the Study
The primary aim of the study was to investigate if neurological assessment can detect cognitive impairment caused by subconcussive trauma in amateur boxing and if possible deficits are related to specific cognitive domains. Secondly, as the first study, we sought to investigate the relationship between neuropsychological evaluation and CSF NFL, previously analyzed in the same cohort .
The study was designed as a prospective prognostic follow-up study. Thirty Olympic boxers competing at high national and/or international level were compared to 25 healthy, age-matched controls. All boxers had completed at least 45 bouts. This inclusion criterion was based on the regulation of the National Boxing Federation demanding an examination including magnetic resonance imaging (MRI), computer tomography (CT) or electroencephalography (EEG) every 50 bouts. The controls consisted of friends or relatives to the boxers, aiming to make certain the enrolment of controls with similar social background and education level similar to the boxers. Exclusion criteria included performance at elite level in sports where head trauma frequently occurs, such a soccer, ice hockey and martial arts. On the same cohort results from biomarker analysis in blood and CSF have been previously published –.
The regional ethical review board at Linköping Health University, Sweden approved the study. Written informed consent was obtained from all participants.
Magnetic Resonance Imaging
Magnetic resonance imaging (MRI) of the brain was performed in all participants without any structural injuries (haemorrhages, subdural haematomas) or other major findings observed.
All participants filled in a questionnaire about medical history, medication, education, present occupation, information about previous concussions and quantification of alcohol and drug intake. Previous sports career was reported to identify those who had trained in sports with risk of TBI. The questionnaire included a 10-question symptom evaluation of head and neck injuries based on a previous study . The number of symptoms that had worsened over the last 5–10 years was combined in a score. The boxers reported about their boxing career; fighting record, number of knock-out losses, number of Referee Stopping Contest losses due to several hard punches to Head (RSC-H), present weight class, duration of career, age at career start and age at first bout , . Boxers gave an account for total amount of bouts the last week prior testing (1–3 bouts) and estimated these bouts as easy (1), intermediate (2) or tough (3). Three boxing experts independently (without knowledge of any results) graded the boxers considering head trauma during total boxing career, 1 to 5 (Type 1 is a boxer that has a low risk to receive blows to the head, according to boxing style, skills and the skills of the opponents. Type 5 is a boxer with high risk to receive repeated blows to the head). The total amount of bouts the last week before test A, the boxers own grading of the bouts and the mean of the expert grading were combined in a score named “Boxing Exposure”. The aim was to calculate the total mild TBI risk prior testing.
All participants underwent a neurological examination by the same physician . The neurological examination protocol included anamnestic questions about concussion symptoms, a general somatic status (general condition, examination of mouth and throat, heart, blood pressure, abdominal palpation, peripheral circulation and skin status) and a neurological status (GCS, orientation, alertness, speech function, cranial nerves 1–12, motor skills, balance testing, coordination, gate, sensibility testing and testing of reflexes).
The cognitive testing was administered at 1–6 days after the last bout (in average 2 days after bout), prior to, but at the same day as CSF and blood tests (test A) . The cognitive testing was performed in a quiet room without distraction at the university hospital at daytime. The same examiner administrated all the tests following a standardized procedure. The duration for the assessment was approximately 60 minutes. A blinded experienced neuropsychologist analyzed the results. Following parts in presented order constituted the test:
1. Rey osterrieth complex figure test, part 1.
Part one involved the untimed reproduction of a complex figure .
2. Vocabulary test.
The task involved the explanation of the meaning of words, ranging from common to less well-known items.
3. Controlled Oral Word Association Test (COWAT).
Participants were asked to generate as many words as possible that begins with a given letter, (i.e. F, A OR S, excluding proper names, numbers or words with different tenses or endings). Sixty seconds was allowed for each letter. The dependent variable was the total number of correct words produced, minus any repetitions .
4. Listening span test.
The participants listened to a set of sentences, half of which were semantically correct and the other half incorrect. Participants were instructed to report whether each sentence was correct or not and to remember the last word in each sentence. This procedure was repeated for two to five sentences. After the sentences had been presented, the participants were asked to recall all the target words in correct order. The task was repeated five times at each level of difficulty .
5. Rey osterrieth complex figure test, part 2.
The second part of the ROCF was included after the Listening Span Test and involved the reproduction (i.e., delayed recall) of the complex figure 30 minutes after the first presentation.
6. Computerized testing: episodic memory – part one.
This test was computerized and constructed by a model from Tulving . Two consecutive lists, one visual and one auditory list containing 36 words each were presented. The visual list was presented on a computer screen and the other list was presented auditory from computer. All words were presented once with interstimulus interval of 2.0 s. Presentation of the words was randomized. After the presentation of the two lists the participants were informed that the test would occur at a later point.
7. Digit span.
Participants were presented with series of numbers, starting with three (for example 3–4–8). There were two series in each level and in total seven levels. The task was to immediately repeat the numbers back. When doing this successfully, the participants were allowed to continue to next level.
8. Trail making test, A and B.
In trail A the participant was instructed to trace a line that connects circled numbers in consecutive order. In trail B the task was to trace a line by alternating circled numbers and circled letters in consecutive order (1–A–2–B–3–C, and so on) .
9. Computerized testing: simple and complex reaction time.
The test was computerized and constructed by a standardized model . In our version of the task, participants were presented with one of two geometrical figures (a circle or a triangle). The size of the stimuli was approximately 80×80 mm. In the condition involving simple reaction time, participants were instructed to respond as quickly as possible whenever the circle appeared on the screen. Complex reaction time required participants to respond with right index finger to the circle and left index finger to the triangle. Each stimulus was presented for 100 msec. A randomly varying interstimulus interval (ISI) was used, ranging between 300 and 5000 msec. The measurement was based on 40 repetitions of each condition. Individual mean values for the simple and complex reaction time were calculated and the difference between complex and simple reaction time used in the further analysis of the results.
10. Computerized testing: Finger Tapping.
The participants were asked to keep their right hand palm down; fingers extended, and rest the index finger on a designated key (the space bar) on a computer keyboard. The participants were instructed to press the key as many times and as fast as possible until a brief pause was introduced. The entire session consisted of five consecutive trials of 15 seconds each with a 15 second rest in between trials. We used the pace ( = mean number of finger taps across trials) in our further analyses.
11. Computerized testing: episodic memory – part two.
Following the part one of the Episodic Memory task, a self-paced, computerized, yes-no recognition test took place. In this test, the words from the two previous word lists were presented, together with distracting words. That is, half the target words were previously listed in the free recall task, half of the target words were uncontaminated by previous attempts at free recall.
Along with the presentation of each word, the participants were asked whether the word had occurred in the previous study lists. When an affirmative answer was given, the subject was also asked to decide if the recognition was accompanied by recollection. If a yes response was connected to an explicit recollection from the presentation earlier, the subject was asked to provide a remember response. The participants were informed that recollections could be associations that took place during the previous presentation, feelings or thoughts that linked the affirmative recognition decision to the previous presentation of the specific word. On the other hand, if the yes-response was not associated with a specific memory (i.e., the decision was based upon another criteria than explicit recollection, such as familiarity or ease of perceptual fluency), the subject was asked to provide a know response. Written instructions explaining the nature of the recollection classification task (including particular examples) were presented on the computer screen during the recognition test. In addition, the experimenter provided additional examples and explanations when the subject found the distinction between knowing and remembering difficult to use.
When the word recognition task was completed followed another word recognition task, also a self-paced computerized, yes-no recognition test. In this test, the participant was asked to discriminate and identify earlier presented audile words among earlier visual presented words and fifteen new distracters. Along with the presentation of each word, the participant was asked to decide whether the word had occurred in the previous study lists as audile stimuli. When an affirmative answer was given, the subject was also asked to decide if the recognition was accompanied by recollection. If a yes response was connected with an explicit recollection from the presentation earlier on, the subject was asked to provide a remember response. On the other hand, if the yes-response was not associated with a specific memory (i.e., the decision was based upon another criteria than explicit recollection, such as familiarity or ease of perceptual fluency), the subject was asked to provide a know response. Also in this task written instructions explaining the nature of the recollection classification task (including particular examples) were presented on the computer screen during the recognition test.
Investigation of Relationship between Neurological Assessment and CSF NFL
Boxers with NFL concentrations elevated more than 2 SD above the control mean, were considered to have abnormal levels.
Statistical analysis was carried out with IBM SPSS Statistics 21.0. Comparisons between groups were performed using the non-parametric Mann-Whitney U-test, due to sample sizes and since some of variables had skewed distributed data, following an initial MANOVA (multivariate analysis of variance). When variables did not meet criteria for conventional MANOVA procedures, the initial MANOVA was carried out on ranked results .
Questionnaire Design and Neurological Examination
The questionnaire about medical and social history and the 10- question survey were similar between boxers and controls . None of the boxers suffered from loss of consciousness during their last bout before test A. Only one of the boxers reported concussion related symptoms after the bout (in this case headache) at the clinical examination, but the medical and neurological examination was normal in all subjects, GCS 15.
Criteria for statistical significance was not reached with an initial MANOVA encompassing neuropsychological results, except for the tests Rey-Osterrieth Figure - part two and Auditory Know HR (F(14,40) = 1.04, p = 0.44). These two did not fulfill criteria for the use of conventional MANOVA. MANOVA of ranked data failed to disclose a statistically significant effect (L(2) = 0.11, p = 0.74). Univariate results are shown in Table 1.
Episodic, language, semantic and working memory.
The scoring in the Rey-Osterrieth Figure Test was made according to established criteria developed by David Loring . Nor the Rey-Osterrieth Figure test or “the delay and episodic memory computer task” revealed any impairment in the boxers compared to controls. In one condition (Visual – Remember HR) the boxers presented with a larger proportion of ‘know’ responses (p = 0.02) than the control group (Table 1).
The test parameters Vocabulary and COWAT showed no differences between the groups (Table 1).
Digit Span and Listening Span were used to assess working memory. No differences were seen in Digit Span but the boxers performed better than the matched controls in the Listening Span Task (p = 0.049, Table 1).
Processing speed and executive functions.
Trail Making, Reaction Time and Finger Tapping were used to evaluate processing speed and executive functions. No differences between the groups were detected in any of those tests.
Visuospatial ability: rey-osterrieth figure test, part one.
Performance on ROCF was similar in boxers and controls (Table 1).
Investigation of Relationship between Neuropsychological Assessment and CSF NFL
Criteria for statistical significance was not reached with an initial MANOVA encompassing neuropsychological results, except for the tests Trail Making B and Simple Reaction Time. (F (14) = 1.4, p = 0.26). These two did not fulfill criteria for the use of conventional MANOVA. MANOVA of ranked data indicated a statistically significant effect (L(2) = 5.4, p = 0.02). Univariate results are shown in Table 2.
Differences in the neuropsychological test results were seen between boxers with abnormal, as compared to normal, CSF NFL at test B (follow-up sample after rest from boxing). Boxers with prolonged NFL elevation had deficits regarding Trail Making A (p = 0.041) and simple reaction time (p = 0.042) (table 2). Comparisons for NFL in closer relation to bout (test A) revealed no statistical significances.
None of the neuropsychological assessment tools could reveal any impairment in the boxers, when compared to controls, although our test battery included attention, processing speed, working memory and visual learning; all the areas where impairment after mild TBI has been previously demonstrated . The performance in some of the boxers was very poor, but similar results were also found among controls. In most tasks the performance was identical or slightly in favor for the boxers.
We have previously in the same cohort of boxers been able to show that the repetitive subconcussive trauma in amateur boxing causes axonal injury detectable by CSF biomarkers , . This is the first study to investigate the relationship between neuropsychological evaluation and CSF NFL (an axonal injury biomarker , ), and the results were quite interesting.
Amateur boxers still having increased CSF NFL concentrations after a rest period of at least 14 days, had significantly worse performance on two (out of four) tasks involving processing speed responses, Trail Making part A (p = 0.04) and Simple Reaction Time (p = 0.04), compared to the other boxers. The boxers with prolonged elevation of CSF NFL had most likely received more punches during bout leading to greater degree of axonal injury, compared to the other boxers , . Another interpretation is that these boxers may suffer from ongoing neurodegeneration. This interpretation may be speculative but in our mind not irrelevant in the light of recent data on progressive neurodegeneration following repeated concussions . The association of CSF NFL with speed measures gives a functional correlate to the CSF biomarker findings and calls for further investigations with repeated CSF samplings and long-term clinical and neuropsychological follow-up within this group.
Our study has some limitations: First, the absence of baseline neuropsychological assessment before bouts may make it hard to determine if the test scores reflect the actual boxing bout. Second, since several statistical tests were performed, there is a risk of type I errors, why the observed relationship between processing speed and prolonged increase in CSF NFL concentration need to be repeated in independent studies before stronger conclusions can be drawn. Whether baseline testing has greater diagnostic accuracy than post concussion assessment alone still remains to be determined , since also the test-retest reliability must be taken into account  and findings further suggest that post injury neuropsychological test data are robust and may not require baseline testing, as long as there exist appropriate, well-developed normative data . However, traditional test norms are based upon quota sampling, the recruitment of a normative sample reflecting the demographic properties of the national census. A problem in investigations involving clinical patients, ethnical minorities, or groups with special skills (such as the athletes in our study) is that these populations often do not reflect the national census in crucial aspects. Hence, census-based norms may bias the interpretation of results in unpredictable ways. In our study, such a method would have overestimated the effects of boxing upon language and possibly episodic memory. Over the years, several different methods for correcting recruitment bias have been suggested . We tried to cope with this problem by recruiting a group of scrupulously matched controls. It should be noted, that this approach in the future should be supplemented with norms derived from meta-analyses of boxer studies. Using Monte Carlo simulation, census-based norms can then be adjusted to reflect recruitment bias. Such a necessary enterprise requires the publication of a larger number of studies than currently available.
The clinical value and validity of neuropsychological assessment in the diagnosis and monitoring of mild TBI has been under discussion. Until now, neuropsychological evaluation has been advocated as the most sensitive tool for detecting early neurological pathology, but our results indicate that without baseline testing and in absence of clinical symptoms, neuropsychological assessment cannot detect the small axonal injuries caused by the repetitive subconcussive trauma in boxing. However, specific test of processing speed and executive functions may indicate protracted or delayed injury and may hold promise as markers for more severe trauma. The clinical relevance of elevated CSF NFL without clinical symptoms or without cognitive impairment can be discussed, but earlier studies have shown that concussion increases the risk for additional concussions , that at least 12% of the athletes with fatal sport-related intracerebral bleeding have suffered from a concussion within 4 weeks prior to the injury  and that sport-related concussion is associated with long-term effects in form of chronic traumatic encephalopathy . The results of these studies indicate that the risk for complications increases if athletes return to sport before the traumatic brain injury has healed and that it today is difficult to decide correctly when that is the case. Just as in fracture healing, absence of clinical symptoms might not be equivalent with absence of/healed traumatic brain injury, why more sensitive analysis tools, such as analysis of CSF brain injury biomarkers, might be helpful for the clinician.
In the future, it would be valuable also investigate the relationship between CSF NFL and some of the established neuropsychological computerized concussion batteries of today (Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), Axon Sports, the Automated Neuropsychological Assessment Metrics (ANAM), Headminder (ImPACT Applications, Inc; Axon Sports, LLC)) in concussed athletes.
This is to our knowledge the first study showing traumatic axonal brain injury can be present without measureable cognitive impairment. Absence of clinical symptoms/cognitive impairment after concussion does not seem to be equivalent to absence of brain injury. Our conclusion is that assessment of memory tests, tests of processing speed or executive functions, cannot detect the small axonal injuries that can be diagnosed with CSF NFL analysis in amateur boxing. However, these modalities can provide complementary information and link CSF biomarker results to functional outcome.
The authors want to thank Lisbeth Hjälle for her helpful assistance with this study.
Conceived and designed the experiments: SN TK JM. Performed the experiments: SN. Analyzed the data: SN HB JM HZ KB TK. Contributed reagents/materials/analysis tools: SN TK KB HZ. Wrote the paper: SN HB JM KB HZ TK.
- 1. McCrory P, Meeuwisse WH, Aubry M, Cantu B, Dvorak J, et al. (2013) Consensus statement on concussion in sport: the 4th International Conference on Concussion in Sport held in Zurich, November 2012. British journal of sports medicine 47: 250–258.
- 2. Gessel LM, Fields SK, Collins CL, Dick RW, Comstock RD (2007) Concussions among United States high school and collegiate athletes. Journal of athletic training 42: 495–503.
- 3. McCrea M, Hammeke T, Olsen G, Leo P, Guskiewicz K (2004) Unreported concussion in high school football players: implications for prevention. Clin J Sport Med 14: 13–17.
- 4. Iverson GL (2005) Outcome from mild traumatic brain injury. Current opinion in psychiatry 18: 301–317.
- 5. Meythaler JM, Peduzzi JD, Eleftheriou E, Novack TA (2001) Current concepts: diffuse axonal injury-associated traumatic brain injury. Archives of physical medicine and rehabilitation 82: 1461–1471.
- 6. Browne KD, Chen XH, Meaney DF, Smith DH (2011) Mild traumatic brain injury and diffuse axonal injury in swine. Journal of neurotrauma 28: 1747–1755.
- 7. Kirov II, Tal A, Babb JS, Lui YW, Grossman RI, et al. (2013) Diffuse axonal injury in mild traumatic brain injury: a 3D multivoxel proton MR spectroscopy study. Journal of neurology 260: 242–252.
- 8. Bianco M, Loosemore M, Daniele G, Palmieri V, Faina M, et al. (2013) Amateur boxing in the last 59 years. Impact of rules changes on the type of verdicts recorded and implications on boxers' health. British journal of sports medicine 47: 452–457.
- 9. Neselius S, Brisby H, Theodorsson A, Blennow K, Zetterberg H, et al. (2012) CSF-biomarkers in Olympic boxing: diagnosis and effects of repetitive head trauma. PloS one 7: e33606.
- 10. Neselius S, Zetterberg H, Blennow K, Randall J, Wilson D, et al. (2013) Olympic boxing is associated with elevated levels of the neuronal protein tau in plasma. Brain injury: [BI] 27: 425–433.
- 11. Neselius S, Zetterberg H, Blennow K, Marcusson J, Brisby H (2013) Increased CSF Levels of Phosphorylated Neurofilament Heavy Protein following Bout in Amateur Boxers. PLOS one 8: e81249.
- 12. Zetterberg H, Hietala MA, Jonsson M, Andreasen N, Styrud E, et al. (2006) Neurochemical aftermath of amateur boxing. Arch Neurol 63: 1277–1280.
- 13. Collins MW, Grindel SH, Lovell MR, Dede DE, Moser DJ, et al. (1999) Relationship between concussion and neuropsychological performance in college football players. JAMA: the journal of the American Medical Association 282: 964–970.
- 14. Hinton-Bayre AD, Geffen G (2002) Severity of sports-related concussion and neuropsychological test performance. Neurology 59: 1068–1070.
- 15. Macciocchi SN, Barth JT, Alves W, Rimel RW, Jane JA (1996) Neuropsychological functioning and recovery after mild head injury in collegiate athletes. Neurosurgery 39: 510–514.
- 16. Echemendia RJ, Putukian M, Mackin RS, Julian L, Shoss N (2001) Neuropsychological test performance prior to and following sports-related mild traumatic brain injury. Clinical journal of sport medicine: official journal of the Canadian Academy of Sport Medicine 11: 23–31.
- 17. Rohling ML, Binder LM, Demakis GJ, Larrabee GJ, Ploetz DM, et al. (2011) A meta-analysis of neuropsychological outcome after mild traumatic brain injury: re-analyses and reconsiderations of Binder, et al. (1997), Frencham, et al. (2005), and Pertab, et al. (2009). The Clinical neuropsychologist 25: 608–623.
- 18. Wall SE, Williams WH, Cartwright-Hatton S, Kelly TP, Murray J, et al. (2006) Neuropsychological dysfunction following repeat concussions in jockeys. Journal of neurology, neurosurgery, and psychiatry 77: 518–520.
- 19. Bleiberg J, Cernich AN, Cameron K, Sun W, Peck K, et al. (2004) Duration of cognitive impairment after sports concussion. Neurosurgery 54: 1073–1078 discussion 1078–1080.
- 20. Bleiberg J, Kane RL, Reeves DL, Garmoe WS, Halpern E (2000) Factor analysis of computerized and traditional tests used in mild brain injury research. The Clinical neuropsychologist 14: 287–294.
- 21. Butler RJ (1994) Neuropsychological investigation of amateur boxers. British journal of sports medicine 28: 187–190.
- 22. Heilbronner RL, Bush SS, Ravdin LD, Barth JT, Iverson GL, et al. (2009) Neuropsychological consequences of boxing and recommendations to improve safety: a National Academy of Neuropsychology education paper. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists 24: 11–19.
- 23. Porter MD (2003) A 9-year controlled prospective neuropsychologic assessment of amateur boxing. Clinical journal of sport medicine: official journal of the Canadian Academy of Sport Medicine 13: 339–352.
- 24. Lezak MD (1984) Neuropsychological assessment in behavioral toxicology–developing techniques and interpretative issues. Scandinavian journal of work, environment & health 10 Suppl 125–29.
- 25. Cornell DG, Roberts M, Oram G (1997) The Ray-Osterrieth Complex Figure Test as a neuropsychological measure in criminal offenders. Arch Clin Neuropsychol 12: 47–56.
- 26. Bigler ED, Rosa L, Schultz F, Hall S, Harris J (1989) Rey-Auditory Verbal Learning and Rey-Osterrieth Complex Figure Design performance in Alzheimer's disease and closed head injury. Journal of clinical psychology 45: 277–280.
- 27. Tulving E (1985) Memory and consciousness. Canadian Psychologist 25: 1–12.
- 28. Wright MJ, Schmitter-Edgecombe M, Woo E (2010) Verbal memory impairment in severe closed head injury: the role of encoding and consolidation. Journal of clinical and experimental neuropsychology 32: 728–736.
- 29. Himanen L, Portin R, Isoniemi H, Helenius H, Kurki T, et al. (2006) Longitudinal cognitive changes in traumatic brain injury: a 30-year follow-up study. Neurology 66: 187–192.
- 30. Wechsler D (1981) Wechsler Adult Intelligence Scale – Revised. Psychological Corporation, New York.
- 31. Benton AL, Hamsher Kd, Sivan AB (1994) Multilingual aphasia examination. Iowa City: AJA Associates, Inc.
- 32. Ross TP (2003) The reliability of cluster and switch scores for the Controlled Oral Word Association Test. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists 18: 153–164.
- 33. Baillargeon A, Lassonde M, Leclerc S, Ellemberg D (2012) Neuropsychological and neurophysiological assessment of sport concussion in children, adolescents and adults. Brain injury: [BI] 26: 211–220.
- 34. Malek-Ahmadi M, Small BJ, Raj A (2011) The diagnostic value of controlled oral word association test-FAS and category fluency in single-domain amnestic mild cognitive impairment. Dementia and geriatric cognitive disorders 32: 235–240.
- 35. Malm J, Kristensen B, Karlsson T, Carlberg B, Fagerlund M, et al. (1998) Cognitive impairment in young adults with infratentorial infarcts. Neurology 51: 433–440.
- 36. Haut MW, Kuwabara H, Leach S, Arias RG (2000) Neural activation during performance of number-letter sequencing. Applied neuropsychology 7: 237–242.
- 37. Kane MJ, Engle RW (2002) The role of prefrontal cortex in working-memory capacity, executive attention, and general fluid intelligence: an individual-differences perspective. Psychonomic bulletin & review 9: 637–671.
- 38. Rosen VM, Bergeson JL, Putnam K, Harwell A, Sunderland T (2002) Working memory and apolipoprotein E: what's the connection? Neuropsychologia 40: 2226–2233.
- 39. Mayers LB, Redick TS, Chiffriller SH, Simone AN, Terraforte KR (2011) Working memory capacity among collegiate student athletes: effects of sport-related head contacts, concussions, and working memory demands. Journal of clinical and experimental neuropsychology 33: 532–537.
- 40. Vallat-Azouvi C, Weber T, Legrand L, Azouvi P (2007) Working memory after severe traumatic brain injury. Journal of the International Neuropsychological Society: JINS 13: 770–780.
- 41. Reitan RM, Wolfson D (1992) Neuropsychological evaluation of older children. Tucson, AZ: Neuropsychology Press.
- 42. Atchison TB, Sander AM, Struchen MA, High WM Jr, Roebuck TM, et al. (2004) Relationship between neuropsychological test performance and productivity at 1-year following traumatic brain injury. The Clinical neuropsychologist 18: 249–265.
- 43. Lange RT, Iverson GL, Zakrzewski MJ, Ethel-King PE, Franzen MD (2005) Interpreting the trail making test following traumatic brain injury: comparison of traditional time scores and derived indices. Journal of clinical and experimental neuropsychology 27: 897–906.
- 44. Perianez JA, Rios-Lago M, Rodriguez-Sanchez JM, Adrover-Roig D, Sanchez-Cubillo I, et al. (2007) Trail Making Test in traumatic brain injury, schizophrenia, and normal ageing: sample comparisons and normative data. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists 22: 433–447.
- 45. Kontos AP, Covassin T, Elbin RJ, Parker T (2012) Depression and Neurocognitive Performance After Concussion Among Male and Female High School and Collegiate Athletes. Archives of physical medicine and rehabilitation 93(10): 1751–1756.
- 46. Haaland KY, Temkin N, Randahl G, Dikmen S (1994) Recovery of simple motor skills after head injury. Journal of clinical and experimental neuropsychology 16: 448–456.
- 47. Prigatano GP, Borgaro SR (2003) Qualitative features of finger movement during the Halstead finger oscillation test following traumatic brain injury. Journal of the International Neuropsychological Society: JINS 9: 128–133.
- 48. Murelius O, Haglund Y (1991) Does Swedish amateur boxing lead to chronic brain damage? 4. A retrospective neuropsychological study. Acta Neurol Scand 83: 9–13.
- 49. Jordan BD (1996) Acute and chronic brain injury in United States National Team Soccer Players. Am J Sports Med 24: 704–705.
- 50. Kaste M, Kuurne T, Vilkki J, Katevuo K, Sainio K, et al. (1982) Is chronic brain damage in boxing a hazard of the past? Lancet 2: 1186–1188.
- 51. Guskiewicz KM, McCrea M, Marshall SW, Cantu RC, Randolph C, et al. (2003) Cumulative effects associated with recurrent concussion in collegiate football players: the NCAA Concussion Study. JAMA 290: 2549–2555.
- 52. Daneman M, Carpenter PA (1980) Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior 4: 450–466.
- 53. Reitan RM (1994) Ward Halstead's contributions to neuropsychology and the Halstead-Reitan Neuropsychological Test Battery. Journal of clinical psychology 50: 47–70.
- 54. Hannay HJ (1986) Experimental techniques in human neuropsychology. New York: Oxford University Press.
- 55. Spreen O, Strauss E (1998) A Compendium of neuropsychological tests: Administration, norms, and commentary, 2nd ed. New York: Oxford University Press.
- 56. Halstead WC (1947) Brain and Intelligence. A Quantitative Study of the Frontal Lobes. Chicago, IL: University of Chicago Press.
- 57. Thomas JR, Nelson JK, Thomas KT (1999) A generalized rank-order method for nonparametric analysis of data from exercise science: a tutorial. Res Q Exerc Sport 70: 11–23.
- 58. Loring DW, Bauer RM (1990) Testing the limits: cautions and concerns regarding the new Wechsler IQ and Memory scales. Neurology 74: 685–690.
- 59. Maruff P, Thomas E, Cysique L, Brew B, Collie A, et al. (2009) Validity of the CogState brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Archives of clinical neuropsychology: the official journal of the National Academy of Neuropsychologists 24: 165–178.
- 60. McKee AC, Stern RA, Nowinski CJ, Stein TD, Alvarez VE, et al. (2013) The spectrum of disease in chronic traumatic encephalopathy. Brain: a journal of neurology 136: 43–64.
- 61. Randolph C, Kirkwood MW (2009) What are the real risks of sport-related concussion, and are they modifiable? Journal of the International Neuropsychological Society: JINS 15: 512–520.
- 62. Comper P, Hutchison M, Magrys S, Mainwaring L, Richards D (2010) Evaluating the methodological quality of sports neuropsychology concussion research: a systematic review. Brain injury: [BI] 24: 1257–1271.
- 63. Echemendia RJ, Bruce JM, Bailey CM, Sanders JF, Arnett P, et al. (2012) The utility of post-concussion neuropsychological data in identifying cognitive change following sports-related MTBI in the absence of baseline data. The Clinical neuropsychologist 26: 1077–1091.
- 64. Mitrushina M, Boone K, D'Elia L (2005) Handbook of normative data for neuropsuchological assessment. New York: Oxford University Press.
- 65. Thomas M, Haas TS, Doerer JJ, Hodges JS, Aicher BO, et al. (2011) Epidemiology of sudden death in young, competitive athletes due to blunt trauma. Pediatrics 128: e1–8.