https://orcid.org/0000-0002-9144-092X
Figures
Abstract
Objective
This study used converging methods to examine the neural substrates of cognitive ability in middle-aged and older men with well-controlled HIV infection.
Methods
Seventy-six HIV+ men on antiretroviral treatment completed an auditory oddball task and an inhibitory control (Simon) task while time-locked high-density EEG was acquired; 66 had usable EEG data from one or both tasks; structural MRI was available for 43. We investigated relationships between task-evoked EEG responses, cognitive ability and immunocompromise. We also explored the structural correlates of these EEG markers in the sub-sample with complete EEG and MRI data (N = 27).
Results
EEG activity was associated with cognitive ability at later (P300) but not earlier stages of both tasks. Only the oddball task P300 was reliably associated with HIV severity (nadir CD4). Source localization confirmed that the tasks engaged partially distinct circuits. Thalamus volume correlated with oddball task P300 amplitude, while globus pallidus volume was related to the P300 in both tasks.
Interpretation
This is the first study to use task-evoked EEG to identify neural correlates of individual differences in cognition in men living with well-controlled HIV infection, and to explore the structural basis of the EEG markers. We found that EEG responses evoked by the oddball task are more reliably related to cognitive performance than those evoked by the Simon task. We also provide preliminary evidence for a subcortical contribution to the effects of HIV infection severity on P300 amplitudes. These results suggest brain mechanisms and candidate biomarkers for individual differences in cognition in HIV.
Citation: Fernandez Cruz AL, Chen C-M, Sanford R, Collins DL, Brouillette M-J, Mayo NE, et al. (2021) Multimodal neuroimaging markers of variation in cognitive ability in older HIV+ men. PLoS ONE 16(7): e0243670. https://doi.org/10.1371/journal.pone.0243670
Editor: Thalia Fernandez, Universidad Nacional Autonoma de Mexico, MEXICO
Received: November 23, 2020; Accepted: July 12, 2021; Published: July 27, 2021
Copyright: © 2021 Fernandez Cruz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because participants did not consent to this use of the data. The vulnerability of the population and the potentially sensitive nature of the data were also considerations. Data are available for researchers who meet the criteria for access to confidential data; contact the authors or the MUHC Research Ethics Board to arrange access: cae@muhc.mcgill.ca.
Funding: LKF, MJB, NEM, DLC received grant funding from the Canadian Institutes of Health Research (TCO-125272) and its HIV Clinical Trials Network (CTN-273; CTN-026) to carry out this work. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Despite effective viral suppression, cognitive impairment is a frequent concern in persons living with HIV, particularly as they grow older. Prevalence estimates are as high as 30 to 50%. [1, 2], although these may be over-estimates due to selection bias [3]. Executive function, attention, and processing speed are commonly affected in those with cognitive impairment [4, 5]. While typically mild in people taking combined antiretroviral therapy (cART), impairment can nonetheless limit occupational function, quality of life, and medication adherence [6–9], as well as the costs of medical care [10]. The causes and underlying neural mechanisms of cognitive impairment remain unclear. Candidate mechanisms include direct effects of HIV infection on the brain, especially prior to cART, cerebrovascular injury due to metabolic complications of chronic HIV, amongst others. There is no consensus on optimal neuroimaging markers or intervention targets.
Recent MRI studies have reported smaller brain volumes for subcortical nuclei and cortical regions in people with HIV infection, compared to HIV- groups [11–13]. However, the patterns of structural differences and their relationships with cognitive impairment have been inconsistent across studies [14] (Chang & Shukla, 2018). Whether brain dysfunction in HIV and the resulting cognitive impairment are the manifestations of processes affecting the whole brain or due to specific dysfunction in vulnerable regions or circuits has yet to be firmly established.
Structural imaging alone is unlikely to provide the answer. In contrast to neurodegenerative disorders such as Alzheimer’s disease, many of the candidate pathophysiological mechanisms in HIV may not cause frank neuronal loss. Cytokine release due to neuroinflammation, for example, might impair synaptic plasticity or interfere with neurotransmission, without yielding macroscopic brain atrophy [15]. While there have been studies of brain function in HIV using functional MRI, less has been done with the simpler and more accessible functional imaging method of EEG [16]. Paired with appropriate cognitive tasks, EEG can assess specific neural circuit function with excellent temporal resolution. EEG is non-invasive and relatively inexpensive, so could be feasible even in resource-poor settings. There is evidence, albeit mainly from small samples, that measures from EEG (or the related technique of magnetoencephalography (MEG)) can distinguish HIV+ from HIV- groups, as well as groups with and without HIV-associated neurocognitive disorders (HAND) [16–19]. However, little work has been done to identify EEG markers associated with individual differences in cognitive performance amongst people with HIV; such markers could shed light on the underlying pathophysiology of cognitive impairment and serve as candidates for diagnostic or intervention-monitoring purposes at the individual level.
Here, we recorded stimulus-locked high-density EEG during performance of two cognitive tests: the Simon task to probe inhibitory control, and a demanding auditory oddball task requiring attention, vigilance and working memory, in the same sample of older men with well-controlled HIV infection. Structural MRI was also available in a sub-sample. In both tasks, we distinguished earlier EEG activity reflecting initial cortical sensory processing and conflict detection and later activity reflecting higher-order cognitive processes [20].
We aimed to provide evidence of the neural mechanisms underlying variation in cognitive performance in the normal to mild cognitive impairment range in men with HIV, seeking evidence for specificity at the level of neural circuits and earlier vs. later stages of cognitive processing. Demographic, metabolic, and HIV infection severity effects on the EEG markers were tested. In a sub-sample, structural MRI was used to localize cortical sources of EEG activity and to explore whether EEG results were explained by structural variation in candidate brain regions.
Materials and methods
Participants
Eighty-nine people living with HIV drawn from the Positive Brain Health Now (BHN) cohort participated in this study. BHN is a longitudinal study of brain health in people over age 35 living with HIV in Canada. Exclusion criteria were dementia precluding informed consent (Memorial Sloan Kettering severity ≥ 3 [21]), current substance dependence or abuse, psychotic disorder or non-HIV related neurological disorder, or terminal illness. Demographic and clinical data were acquired by self-report questionnaires, chart review and a brief cognitive test battery [22, 23]. We acquired high-density EEG in participants in two intervention sub-studies (trials of cognitive training or physical exercise) that drew from the BHN cohort (Fig 1). EEG was acquired as part of the baseline assessment prior to randomization into these intervention trials. These sub-samples were randomly drawn from BHN participants in Montreal, based on scores on a cognitive assessment battery. Sixty-nine participants were drawn from those with cognitive scores below the median, 20 from those above the median. Structural MRI was also available in 58 people in this sample. Although the BHN study included women, too few women (n = 6) were recruited into the intervention sub-studies (and thus completed EEG or MRI) to allow meaningful analysis by sex. Based on other work in this cohort, we expect differences in the causes and manifestations of brain dysfunction in men and women in this Canadian sample [9]. We thus report only on men here, to avoid falsely generalizing to women. The protocol was approved by the ethics board of the McGill University Health Centre and all participants provided written informed consent.
Participants who performed the task at chance accuracy or who had fewer than 25 usable EEG trials after pre-processing were excluded. Data were only available for 6 women, insufficient to reliably adjust for sex effects, so the analysis was restricted to men. Sixty-six men had complete data for at least one task, 47/66 had complete data for both tasks. Structural MRI was available in 43/66. Cognitive performance data were available for all participants.
Cognitive and electrophysiological assessments
Auditory oddball task.
The multiple auditory oddball task is shown in Fig 2. On each of 260 trials, participants fixated a cross on the computer screen and heard a series of 4 tones, the third of which might differ in frequency slightly (small deviant) or substantially (large deviant) from the other three. Participants reported whether this tone was the same or different with a left- or right-hand button press. Participants had a practice block with 28 trials before EEG recording began. The sensitivity index d’ was calculated for the large and small deviant tones with the formula d′ = hit rate−false alarm rate to estimate discrimination accuracy. Audiometry was carried out to ensure hearing adequate to perform the task.
Upper panel shows an example trial of the multiple auditory oddball task and the grand average topographic maps for the P300 with the 2 clusters of electrodes used for analysis highlighted (A and B). Participants completed 260 trials, each consisting of a sequence of four 200-milliseconds long tones presented binaurally at 70dB. The first, second and last tone of each of the trials were always “standard” tones (1047 Hz), the third tone could be either a standard tone or one of two deviant tones; a “small deviant” tone (1078 Hz) or a “large deviant” tone (1175 Hz). Participants were instructed to fixate a cross in the middle of the computer screen, and to report whether the third tone of each of the series was the same (right button press) or different (left button press) from the other three tones. Of the 260 trials, 146 (about 56% of all trials) were standard trials, 57 were small-deviant trials and 57 were large-deviant trials (each about 22% of all trials), presented in random order. (A) Grand average over the electrode cluster surrounding Cz and (B) over a cluster chosen following a data-driven approach.
Simon task.
The Simon task required participants to make a left or right button press response according to an arbitrary sensorimotor rule: one of two shape cues was associated with the left button, the other with the right button (randomized across participants). The shape appeared on the left or right side of the screen, inducing a pre-potent response tendency to respond with that hand. The cue and side of presentation matched on congruent trials (190) and conflicted on incongruent trials (190) (Fig 3). Sixteen practice trials preceded the main task. Reaction time (RT) and accuracy were recorded.
Upper panel shows example trials of the Simon task and the grand average topographic maps for the P300 with the two clusters of electrodes used for analysis highlighted (A and B). During the Simon task (upper right panel) participants responded to one of two 2.5 x 2.5 cm geometrical shapes (square or diamond) mapped to either a left or a right button press (counterbalanced across participants). Participants were instructed to press the button associated with the stimulus, regardless of where it appeared. There were 380 trials, 190 congruent (side of presentation and required response the same) and 190 incongruent (required response opposite to the side of presentation), presented in random order. Following the motor response, there was an inter-trial interval lasting 1 to 1.5 seconds (ITI). (A) Grand average over the electrode cluster surrounding Cz and (B) over a cluster chosen following a data-driven approach.
Cognitive assessment.
Cognitive ability was assessed with the Brief Cognitive Ability Measure (B-CAM), a computerized battery developed to assess domains typically affected in people with HIV, including processing speed, attention and cognitive control (Flanker task, Corsi block task), episodic memory (verbal list learning), and phonemic fluency (FAS). We have shown that cognition can be assessed as a unidimensional construct in HIV [22, 24]. The B-CAM was developed using item-response theory as a measure of this latent construct. This battery has been validated against real world function (work status) and its concordance with multiple existing methods of operationalizing the Frascati classification of HAND has been reported [8]. Fifty-three participants included here also took part in another BHN sub-study that administered standard neuropsychological testing, covering six domains, at least two tests per domain [25], allowing HAND classification.
Co-morbidity characterization
Self-reported depression and anxiety symptoms were assessed with the Hospital Anxiety and Depression Scale [26]. Risk of metabolic syndrome was calculated as a binary variable (risk or no risk). In accordance with the National Cholesterol Education Program (NCEP) Adult Treatment Panel (ATP) III guidelines, risk was defined if three or more diagnostic features of metabolic syndrome were present [27]. Features considered were: abdominal obesity (waist circumference ≥102 cm), elevated triglyceride levels (≥1.7 mmol/L) or on lipid-lowering treatment, reduced HDL-C level (<1.0 mmol/L) or on lipid-lowering treatment, elevated blood pressure (>130/85 mmHg) or on antihypertensive treatment, and elevated fasting glucose (≥5.6 mmol/L) or on medication for diabetes.
Electroencephalographic recordings and analysis
EEG was recorded with a 256-channel HydroCel Geodesic-Sensor-Net (Electrical Geodesics, Inc., Eugene, OR) using Net Station 5 software. Data were collected using the electrode Cz as reference and keeping electrode impedance below 50 kΩ. Seventy-eight electrodes located over the neck and cheeks were excluded before data pre-processing, as these were consistently contaminated with muscle artifacts.
EEG data were analyzed using Brainstorm [28]. Pre-processing followed standard procedures as recommended in [29] and Brainstorm. Continuous EEG recordings were filtered (0.1–30 Hz) down-sampled to 500 Hz and re-referenced to the right and left mastoid electrodes. Automatic blink detection was done using four electrodes located above and below each eye and artifact correction was performed with Signal-Space Projection.
Only correct trials without activity exceeding ± 100 μV were analyzed. The average number of trials included for the Simon task was 109 (SD 34) (congruent), and 102 (SD 38) (incongruent). For the oddball task, a mean of 51 (SD 8) large deviant and 49 (SD 9) small deviant trials were included.
ERP analysis.
Time windows were defined according to the maxima and time distributions of grand average waveform of the conditions of interest for each task. ERPs of interest for the oddball task were the N100 (mean amplitude between 90 and 120ms) and the P300 (mean amplitude between 280–400ms). For the Simon task, the analysis focused on the N200 (mean amplitude between 200–300 ms) and the P300 (mean amplitude between 300–550 ms). All epochs had a 200 ms pre-stimulus baseline.
Data extraction was twofold. First, clusters of electrodes centered on electrodes at which ERPs have a maximum distribution were used. The amplitude of the P300 was extracted at a cluster of electrodes (E081, E045, E132) centered on Cz [30, 31]. The amplitude of the N100 evoked by the oddball task was extracted from the cluster centered at Cz, and the N200 evoked by the Simon task at a cluster centered on frontocentral electrodes (E015 and E023) [32]. Additional extraction was in line with previous studies showing decreased P300 amplitude in cognitively impaired participants [33, 34]. This focused on the parieto-occipital and central-parietal clusters with maximum differences between conditions (corrected with False Discovery Rate at the 0.01 level) (Figs 2 and 3).
MRI acquisition and analysis
MRI was acquired using a 3T Siemens Tim TRIO v17 scanner (Siemens AG, Erlangen, Germany) with a 12-channel transmit/receive head coil. The scanning protocol included a T1-weighted three-dimensional magnetization-prepared rapid acquisition gradient echo sequence (repetition time (TR)/echo time (TE)/inversion time (TI) = 2300/2.98/900ms; voxel = 1.0mm3; flip angle = 9˚). The T1-weighted data were processed as described previously [35]. The pre-processing included denoising [36], intensity inhomogeneity removal [37] and brain extraction [38]. Linear registration of images was initially done in relation to the Montreal Neurological Institute ICBM152 template using a nine-parameter affine transform [39]. Linear registration was followed by a non-linear registration [40]. After registration, brain regions identified on the ICBM152 template [41] were mapped back to the subject’s data to accurately identify specific structures of interest. The volumes in each of these regions were defined as the volume (cm3) of all segmented voxels in the standard space. See also [12] for more details.
Head modeling and ERP source estimation.
ERP source estimation was carried out for the 43 participants who also underwent MRI. FreeSurfer (http://surfer.nmr.mgh.harvard.edu/) was used to extract and register cortical surfaces and white matter envelopes. Head modeling was performed using a symmetric boundary element method as implemented in the OpenMEEG software [42]. Minimum-norm imaging was used to estimate cortical sources of ERPs [43]. Current density maps and dipole orientations constrained to be normal to cortex were used for inverse modeling. Z-score transformations were done to convert current density values at the subject level to a score representing the number of standard deviations with respect to the baseline period. Sources were rectified to absolute values and projected to default anatomy template before averaging across participants. Gaussian smoothing with a 3 mm full width at half maximum was applied to the final averages.
Statistical analyses. Statistical analyses were performed using R version 3.4.2. Repeated measures ANOVA with trial type as within subject factor was used to test for the effects of trial type on the task performance and ERPs of interest (N100, N200 and P300). This was done independently for each task using the factors small vs. large deviant vs. standard for the oddball task and congruent vs. incongruent for the Simon task. For the P300 effects–evaluated at two electrodes–electrode was included as an additional factor. To address our main question on the neural correlates of individual variation in cognitive ability in HIV+ individuals, we performed multiple linear regressions assessing the contribution of ERPs amplitude in predicting overall cognitive ability (B-CAM score).
Similarly, the effects of current and nadir CD4 cell counts on electrophysiological variables were assessed using multiple linear regression. Given that most participants had current CD4 cell counts in the normal range, and in line with recent MRI literature in cART-treated patients, it was hypothesized that the nadir CD4 count would be more relevant in predicting HIV-related EEG effects. Age and education (dichotomized as less than university, or at least some university education) were included in all models. Including educational effects at a finer grain did not contribute further, given the sample size and relatively high level of education of participants.
Pearson correlations were used to test whether volumes of subcortical nuclei previously shown to be affected in HIV and known to be involved in the cognitive tasks studied here (thalamus, globus pallidus and putamen) explained the variance observed in P300 amplitudes. Correlations between P300 amplitude and extra-cerebral cerebrospinal fluid (CSF) and total grey matter volumes were also examined to determine if diminished amplitudes were explained by overall brain volume or non-specific differences in conductance due to expanded CSF space between the cortex and scalp.
Results
Participant characteristics
Demographic and clinical variables are summarized in Table 1. All participants were taking cART and had intact hearing in the frequency range of the stimuli used in the oddball task. In the subset who underwent full neuropsychological testing, those meeting HAND criteria (asymptomatic neuropsychological impairment or mild neurocognitive disorder) had significantly lower BCAM scores (mean = 19.7, SD = 4.2) than those who did not meet HAND criteria (mean = 24.8, SD = 3.5), (t (51) = -3.2, p = 0.002).
Behavioral performance
In the oddball task, large deviants were discriminated faster and more accurately than small deviants. Repeated measures ANOVA with trial type as within-subject factor showed a significant effect of trial type on RT (F (2, 52) = 60.79, p < 0.001, ε = 0.70, and d’ (F (1, 53) = 14.06, p < 0.001, ε = 0.21). Post hoc Bonferroni-corrected pairwise comparisons showed that RTs were faster for large vs. small deviant (t (53) = -7.52, p < 0.001), and vs. standard trials (t (53) = -10.78, p < 0.001), as well as for standard vs. small deviants (t (53) = 4.51, p < 0.001). For the Simon Task, trial type had significant effects on reaction time (F (1, 57) = 80.56, p < 0.001, ε = 0.58) and accuracy (F (1, 57) = 39.99, p < 0.00, ε = 0.41), with participants less accurate and slower on incongruent trials.
Task-evoked potentials
In the oddball task, the factors trial type (F (2, 53) = 42.51, p < 0.0001, ε = 0.45), cluster of electrodes (F (1, 53) = 20.48, p < 0.0001, ε = 0.28), and their interaction (F (2, 53) = 11.11, p < 0.0001, ε = 0.17), had significant effects on the amplitude of the P300. Post-hoc pairwise comparisons (Bonferroni-corrected) showed that the amplitude of the P300 for the large deviant was significantly larger from that of the small deviant and the standard tone (p < 0.0001). The effect of trial type was also significant on the N100 (F (2, 53) = 18.87, p < 0.0001, ε = 0.26), with the N100 evoked by the large deviant significantly different from the small deviant and standard tone (p< 0.0001). Evoked responses for the oddball task are shown in Fig 2.
In the Simon task, neither trial type nor cluster of electrodes had significant effects on the P300 or N200 amplitude. A significant interaction was found between these two factors (F (1, 58) = 9.40, p = 0.003, ε = 0.14), but post-hoc pairwise comparisons showed no significant differences between the P300 amplitude of congruent and incongruent trials. Evoked responses for the Simon task are shown in Fig 3.
Relationship between task performance and cognitive ability
The validity of both tasks as indicators of overall cognitive ability was confirmed using multiple linear regression to predict overall cognitive ability (B-CAM scores). In the oddball task, d’ for the large deviant and age explained variation in B-CAM scores (F (3,50) = 10.3, p < 0.0001). Neither RTs nor d’ for the small deviant predicted cognitive ability. Thus, only large deviant trials were considered for further analysis.
In the Simon task, a significant relationship was found between incongruent trial RT and cognitive ability (F (3,55) = 4.56, p = 0.006). Neither congruent RT nor accuracy predicted B-CAM scores. Hence, the incongruent condition was retained for subsequent analysis.
Relationships between task performance and HIV severity
Multiple linear regression showed that neither CD4 cell count (nadir, current), age nor education predicted performance of either task (all p > 0.1).
Relationships between ERPs and cognitive ability
Multiple linear regression models were fitted for each task to evaluate if neural activity reflecting specific cognitive processes related to overall cognitive ability. B-CAM score was not significantly predicted when the amplitudes of early ERPs (N100 and N200) were included in the models (p > 0.1). In contrast, later ERPs were associated with overall cognitive ability in both tasks. The amplitude of the P300 evoked by the oddball task at the cluster of electrodes around Pz was significantly related to B-CAM (F (3, 50) = 4.73, p = 0.005), with a marginal effect of P300 amplitude (p = 0.1) and age (p = 0.04). At the central parietal cluster, the relationship was also significantly related to B-CAM (F (3, 50) = 6.60, p = 0.0007) (Table 2). In the Simon task, the relationship was similar at the cluster around Cz (F (3, 55) = 5.32, p = 0.002) and replicated at the second parieto-occipital cluster of electrodes: B-CAM score was predicted by the amplitude of the P300 (p = 0.0007) and age (p = 0.008) (F (3, 55) = 8.02, p = 0.0001) (Table 2).
Relationships between ERPs and immunocompromise
For each of the tasks, linear regressions were carried out to test if current or past HIV infection severity contributed to the variation in neural activity in early or late stages of cognitive processing. The amplitudes of early ERPs were not significantly explained by nadir or current CD4 count. Nadir CD4 counts (p = 0.0006) and age (p = 0.0001) independently contributed to predicting the P300 amplitude evoked by the oddball task at the temporal cluster of electrodes, with an increase in 100 cells/μL in the nadir CD4 count related to an increase in amplitude of about 1 μV, (F (3,50) = 11.81, p < 0.000001). A similar significant contribution was found at the cluster centered on Pz (page = 0.007, pnadir_cd4 = 0.05). Adding current CD4 count accounted only for an additional 3% of the variance observed in the P300 amplitude at the temporal cluster of electrodes. Neither nadir nor current CD4 counts explained the amplitude of the P300 evoked by the Simon task at either cluster of interest (p > 0.9) (Table 3; Fig 4).
Partial regression plots of the relationship of P300 amplitudes evoked by (A) oddball and (B) Simon tasks with cognitive ability (B-CAM score) and immunocompromise (nadir CD4 cell count), controlling for the effects of age and education. *** p < 0.001.
We repeated the linear regressions restricted to the participants who had completed both tasks (n = 47) to ensure that the observed differences in the relationship between nadir CD4 count and P300 amplitudes in the two tasks was not due to differences in the participants included in each analysis. The pattern of results remained unchanged; p >0.9 for the Simon task and p = 0.0001 for the oddball task. We additionally tested if depression or anxiety, assessed with the HADS, or presence of metabolic syndrome explained any variance in the ERPs of interest. None of the regressions were statistically significant.
Brain basis of evoked potentials
Fig 5 shows the EEG cortical source estimations for both tasks, in those for whom MRI data were available. Thalamus volume was related to the amplitude of the P300 evoked by the oddball task (oddball P300, n = 27, r = 0.36 p = 0.03) but not to the P300 evoked by the Simon task (p’s > 0.14). The volume of the globus pallidus showed similar relationships with the P300 amplitude of both tasks (oddball P300, n = 27: r = 0.32, p = 0.03, Simon P300, r = 0.46, p = 0.008) (Fig 6). There was no correlation between P300 amplitude evoked by either task and putamen volume, nor were P300 amplitudes related to total grey matter volume or extra-cerebral CSF volume (ps > 0.2).
Source estimation for EEG activity in response to the oddball (A, B) and Simon tasks (C, D). Current density maps for both tasks were normalized in relation to values during the baseline period prior to stimulus presentation (-200 to 0 ms). Z-scores are with respect to the baseline period. (A) Current density maps during early auditory processing in response to the large deviant tone between 70 and 90 ms post stimulation. (B) Current density maps indicating sources for the P300 peak activations (280–400 ms) in response to the large deviant tone. The P300 evoked during the oddball task showed current density foci peaking in middle and inferior temporal gyri, posterior cingulate, temporal-parietal junction and the posterior superior frontal gyrus, principally on the right. (C) Early visual activity between 90–120 ms in response to incongruent stimuli. (D) Peak activations during the P300 time window in response to incongruent stimuli (300–550 ms). Lateralization to the right hemisphere also occurred for the P300 of the Simon task. As in the oddball task, activity localized to the temporal-parietal junction, but otherwise engaged different cortical regions: the anterior cingulate, middle and anterior-superior frontal gyrus and superior temporal gyrus. R = right hemisphere.
Data are from the 27 participants with usable EEG data in both tasks and structural MRI. Asterisks indicate significant correlations (uncorrected). The scale indicates Pearson’s r coefficients.
Discussion
This study of the neural correlates of cognitive ability in middle-aged and older men with well-controlled HIV found evidence for specific vulnerabilities in processing stages and neural circuits. Behavioral measures from two tasks differing in higher-order response control and sensory-attentional requirements were related to overall cognitive ability (across a range from normal to mildly impaired) in this large, well-characterized sample. Stimulus-locked EEG activity evoked by these tasks at early sensory and cognitive processing stages was not significantly related to overall cognitive ability, nor was it associated with past HIV infection severity indicated by nadir CD4 count. In contrast, the amplitude of the P300 waveform evoked by both tasks did relate to cognitive ability assessed by a short battery of neuropsychological tests. In the auditory oddball task, P300 amplitude was also related to nadir CD4 count. Co-morbid depression, anxiety or metabolic syndrome did not explain variation in P300 amplitude in either task and ERP effects were not explained by global brain atrophy in the sub-sample who also underwent structural MRI. Instead, we found preliminary evidence implicating subcortical nuclei in P300 amplitude variation.
Prior smaller studies have compared early potentials evoked by auditory oddball and visual stimulation tasks in HIV+ and HIV- groups. Most reported no difference in N100 or N200 amplitudes between groups [44–46], although two studies found reduced N200 amplitudes in HIV+ groups [44, 47]. No work to date has asked whether early potentials relate to cognitive performance. Here, in a larger sample of HIV + individuals with varying degrees of cognitive ability, we show that these earlier processing stages (100–200 ms) in auditory and visual modalities do not reliably relate to cognitive performance nor to past HIV infection severity.
In contrast, cognitive ability was related to the amplitude of the P300 evoked by both tasks. Prior work on the P300 in HIV, mainly evoked with oddball tasks, found amplitude reductions in HIV+ compared to HIV- groups, in smaller samples with shorter duration of HIV infection than those studied here (reviewed in [16]). Here, taking an individual differences approach in men with HIV infection, we confirm the relevance of the P300 as an indicator of brain dysfunction in HIV, show for the first time that it relates to cognitive ability in this population, and find that this relationship is present across two different cognitive tasks in the same sample. Smaller ERP amplitudes are a well-known feature of aging [48, 49], but the effects reported here were over and above those explained by age.
Neural activity in the P300 time window in the oddball task is thought to reflect attention towards working memory contents [31, 50] and arises primarily from the anterior cingulate, superior temporal gyrus, temporal-parietal junction and prefrontal cortex [51]. The Simon task P300 activity is thought to reflect action selection in the face of conflict, in addition to attention, as the task requires inhibition of prepotent responses [52]. This P300 originates principally from the middle and inferior frontal gyri and the superior parietal lobule [53]. Our source localization findings are largely in line with this literature in healthy subjects. We found that temporal-parietal junction, inferior temporal, pre-central and posterior regions of the superior frontal gyrus as well as posterior cingulate cortex contributed to P300 activity during the oddball task. P300 sources in the Simon task included middle and superior frontal gyrus, anterior cingulate, superior temporal gyrus and temporal-parietal junction. Thus, qualitatively, there were shared and distinct cortical sources for activity in this time window, across tasks, as intended.
The severity of past immunosuppression, indicated by the nadir CD4 count, explained variance in P300 amplitude in the oddball task only. There are likely multiple contributors to cognitive dysfunction in HIV [9]; this result suggests that P300 amplitude generally may be a good indicator of brain health, but the specific neural circuit underlying the auditory oddball task may be more vulnerable to the effects of HIV infection, at least in middle-aged and older men.
Taking advantage of the availability of MRI data for a subset of this sample, we provide preliminary evidence that this may be due to thalamic engagement in this task: smaller thalamus volumes were associated with smaller oddball P300 amplitude. The thalamus contributes to stimulus monitoring for subsequent evaluation according to context and working memory content and is engaged in auditory and visual oddball tasks [54, 55]. Recent neuroimaging studies using MRI, fMRI or PET have shown that the thalamus is amongst the structures most affected in HIV [12, 56–58], although this is not a universal finding [59]. Alternatively, this apparent circuit selectivity could reflect vulnerability of temporal-parietal cortex; the limited sample with MRI data available here does not allow a strong test of the latter possibility. In any case, these results suggest that weaker cognitive performance in HIV reflects more than the fronto-striatal dysfunction that has been a focus in the literature [60].
The relation between oddball P300 and nadir CD4 count, even after good immune recovery, may reflect brain injury sustained at the time of more severe HIV infection, or on-going effects of that injury, a so-called “legacy effect”. Thus, early treatment initiation and on-going effective viral suppression may limit or prevent brain injury in persons living with HIV. Encouragingly, the few longitudinal neuroimaging studies to date show no strong evidence for progressive brain volume loss in cART treated individuals, at least over short periods [57, 61], in line with that idea.
A third explanation for the apparent circuit-specific vulnerability to HIV severity of the oddball task P300 that we observed here is that the waveforms evoked by these two tasks might differ in their measurement properties. While there were no ceiling or floor effects in either task, and the range and standard deviation of P300 amplitudes was similar, it could be that the signal-to-noise ratio of the Simon task-evoked P300 is lower than that of the oddball task, which would make it harder to detect a relationship with nadir CD4 count. Whether the reasons are technical or neurological, our findings argue that the auditory oddball task-evoked P300 may be more suitable for detecting HIV-related brain dysfunction than the P300 evoked by the Simon task, although both reflect cognitive ability.
Attenuation of P300 amplitude in auditory oddball and other tasks has been reported in other cognitively impaired populations, such as Alzheimer’s disease and amnestic mild cognitive impairment [33, 34], indicating that P300 amplitude reduction is not specific for HIV-associated neurocognitive disorders. EEG alone is thus better suited as a marker for brain dysfunction, rather than for identifying specific underlying causes. We also note that the P300 evoked in the present study is of smaller amplitude to those typical of the wider literature. This may be because the multiple oddball task here involves a series of tones, likely requiring greater attention to detect deviants than the conventional single tone task.
After accounting for age and education, nadir CD4 count only reliably explains some of the variance in P300 amplitude in the oddball task, and none in the Simon task. This emphasizes the need for more comprehensive identification of the factors contributing to variation in brain function in HIV. Dysfunction could be due to additional direct effects of HIV not indicated by nadir CD4 count, and many additional candidate mechanisms co-occurring or indirectly related to HIV status. Given sample size, we assessed a limited set of additional candidate mechanisms, finding no effect of anxiety, depression or metabolic risk (as a proxy for cerebrovascular injury) on the EEG measures. There may also be additional resilience factors beyond education [33]; for example, in other work, we have identified engagement at a high level in arts, music, sports or travel as indicators of cognitive reserve in older people with HIV [62].
This study has limitations. Foremost, the findings can be generalized only to middle-aged and older, relatively well-educated HIV+ men, on cART, and without clinically obvious dementia. There is a need to carry out similar work in women living with HIV. As only a subset of the sample had structural MRI, there was insufficient statistical power to carry out whole-brain analyses linking ERP and MRI, which was not an a priori aim of this study. Thus, the observed relationships between subcortical volumes and evoked potentials should be treated as exploratory and require replication in a new sample. Whole brain approaches would provide additional information on possible regional cortical or white matter contributors to the EEG variance, not tested here. Finally, we focused on variability in cognition amongst men with HIV; given this design, which did not include an HIV- group, we cannot claim that effects are HIV-specific, although the effects related to nadir CD4 count presumably reflect HIV-linked processes. In any case, our aim here was to identify EEG indicators of cognition in men with HIV, regardless of whether cognitive performance variability was uniquely determined by HIV status.
Other comorbidities common in HIV + individuals, such as cerebrovascular disease, cART neurotoxicity, lifestyle factors, or social exclusion may also contribute to brain dysfunction [63, 64]. Although the present study is amongst the largest EEG studies ever conducted in HIV, the sample size is nonetheless far from what would be needed to exhaustively test the effects of such factors.
Conclusion
These are the first EEG results contrasting two tasks tapping different cognitive domains (with at least partially distinct underlying neural circuitry) in the same large, well-characterized sample of men with well controlled HIV infection. We showed that evoked potentials in the P300 time window are promising markers of cognitive ability in the mildly impaired-to-normal range of neuropsychological performance and suggest that tasks and EEG measures related to cortico-thalamic circuits may be particularly useful for probing brain health in people living with HIV. The immune effects of HIV at the time of untreated infection, indexed by nadir CD4 count, are related to brain function assessed by EEG even after many years of cART, and specific EEG measures in turn relate to current cognitive status. There appear to be additional, unmeasured contributors to EEG variance in these older men, beyond demographic variables; neither anxiety, depression nor the presence of metabolic syndrome was explanatory here. It is increasingly clear that cognitive impairment in people living with HIV can have many contributors, at least some of which are likely to be treatable or remediable [9, 64, 65]. The findings here suggest that evoked potentials, a cheap and widely available readout of brain function, may prove useful in targeting or monitoring novel treatment or rehabilitation approaches to this common and quality-of-life-limiting aspect of living with HIV.
Acknowledgments
We would like to thank Christine Déry and Susan Scott for their help with participant recruitment and data analysis, Marcus Sefranek for technical assistance and the Positive Brain Health Now investigators, research staff and participants for making this work possible.
References
- 1. Heaton R. K., Clifford D. B., Franklin D. R., Woods S. P., Ake C., Vaida F., et al. (2010). HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology, 75, 2087–2096. pmid:21135382
- 2. Su T., Schouten J., Geurtsen G. J., Wit F. W., Stolte I. G., Prins M., et al. (2015). Multivariate normative comparison, a novel method for more reliably detecting cognitive impairment in HIV infection. Aids, 29(5), 547–557. pmid:25587908
- 3. Brouillette M.-J., Mayo N. E., Fellows L. K. (2018) Estimates of prevalence of cognitive impairment from research studies can be affected by selection bias. JAIDS, 78(2), e7–e8. pmid:29489624
- 4. Goodkin K., Miller E. N., Cox C., Reynolds S., Becker J. T., Martin E., et al. (2017). Effect of ageing on neurocognitive function by stage of HIV infection: evidence from the Multicenter AIDS Cohort Study. The Lancet HIV, 4(9), 411–422. pmid:28716545
- 5. Valcour V., Shikuma C., Shiramizu B., Watters M., Poff P., Selnes O., et al. (2004). Higher frequency of dementia in older HIV-1 individuals: The Hawaii aging with HIV-1 cohort. Neurology, 63(5), 822–827. pmid:15365130
- 6. Becker B. W., Thames A. D., Woo E., Castellon S. A., & Hinkin C. H. (2011). Longitudinal change in cognitive function and medication adherence in HIV-infected adults. AIDS and Behavior, 15(8), 1888–1894. pmid:21437726
- 7. Heaton R. K., Marcotte T. D., Mindt M. R., Sadek J., Moore D. J., Bentley H., et al. (2004). The impact of HIV-associated neuropsychological impairment on everyday functioning. Journal of the International Neuropsychological Society: JINS, 10(3), 317–331. pmid:15147590
- 8. Brouillette M.-J., Koski L., Gasparri J., Brew B.J, Mayo N. E, Fellows L. K., et al. Predicting occupational outcomes from neuropsychological test performance in older people with HIV. AIDS, under review. pmid:33927088
- 9. Mayo N. E., Brouillette M.-J., Scott S. C., Harris M., Smaill F., Smith G., et al. (2019). Relationships between cognition, function, and quality of life among HIV+ Canadian men. Quality of Life Research.
- 10. Yeung H., Krentz H. B., Gill M. J., & Power C. (2006). Neuropsychiatric disorders in HIV infection: impact of diagnosis on economic costs of care. AIDS, 20(16), 2005–2009. pmid:17053346
- 11. Holt J. L., Kraft-Terry S. D., & Chang L. (2012). Neuroimaging studies of the aging HIV-1-infected brain. Journal of Neurovirology, 18, 291–302. pmid:22653528
- 12. Sanford R., Fernandez Cruz A. L., Scott S. C., Mayo N. E., Fellows L. K., Ances B. M., et al. (2017). Regionally Specific Brain Volumetric and Cortical Thickness Changes in HIV-infected Patients in the HAART era. J Acquir Immune Defic Syndr, 74, 563–570. pmid:28129254
- 13. Thompson P. M., & Jahanshad N. (2015). Novel Neuroimaging Methods to Understand How HIV Affects the Brain. Current HIV/AIDS Reports, 12, 289–298. pmid:25902966
- 14.
Chang L., & Shukla D. K. (2018). Imaging studies of the HIV-infected brain. in Handbook of clinical neurology (1st ed., Vol. 152). Elsevier B.V.
- 15. Nolan R., & Gaskill P. J. (2019). The role of catecholamines in HIV neuropathogenesis. Brain Research, 1702, 54–73. pmid:29705605
- 16. Fernández-Cruz A. L., & Fellows L. K. (2017). The electrophysiology of neuroHIV: A systematic review of EEG and MEG studies in people with HIV infection since the advent of highly-active antiretroviral therapy. Clinical Neurophysiology, 128(6), 965–976. pmid:28433855
- 17. Groff B. R., Wiesman A. I., Rezich M. T., O’Neill J., Robertson K. R., Fox H. S., et al. (2020). Age-related visual dynamics in HIV-infected adults with cognitive impairment. Neurology—Neuroimmunology Neuroinflammation, 7(3), e690. pmid:32102916
- 18. Lew B. J., McDermott T. J., Wiesman A. I., O’Neill J., Mills M. S., Robertson K. R., et al. (2018). Neural dynamics of selective attention deficits in HIV-associated neurocognitive disorder. Neurology, 91(20), e1860–e1869. pmid:30333162
- 19. Wiesman A. I., O’Neill J., Mills M. S., Robertson K. R., Fox H. S., Swindells S., et al. (2018). Aberrant occipital dynamics differentiate HIV-infected patients with and without cognitive impairment. Brain, 141(6), 1678–1690. pmid:29672678
- 20. Luck S. J., Woodman G. F., & Vogel E. K. (2000). Event-related potential studies of attention. Trends in Cognitive Sciences, 4(11), 432–440. pmid:11058821
- 21. Price R., & Brew B. (1988). The AIDS dementia complex. J Infect Dis., 158(5), 1079–1083. pmid:3053922
- 22. Brouillette M. J., Fellows L. K., Finch L., Thomas R., & Mayo N. E. (2019). Properties of a brief assessment tool for longitudinal measurement of cognition in people living with HIV. PLoS ONE, 14(3), 1–13. pmid:30908501
- 23. Mayo N. E., Brouillette M.-J., & Fellows L. K. (2016). Understanding and optimizing brain health in HIV now: Protocol for a longitudinal cohort study with multiple randomized controlled trials. BMC Neurology, 16:8. pmid:26762403
- 24. Koski L., Brouillette M. J., Lalonde R., Hello B., Wong E., Tsuchida A., et al. K. (2011). Computerized testing augments pencil-and-paper tasks in measuring HIV-associated mild cognitive impairment. HIV Medicine, 12(8), 472–480. pmid:21395965
- 25. Antinori A., Arendt G., Becker J. T., Brew B. J., Byrd D. A., Cherner M., et al. Wojna V. E. (2007). Updated research nosology for HIV-associated neurocognitive disorders. Neurology, 69, 1789–1799. pmid:17914061
- 26. Zigmond A. S., & Snaith R. P. (1983). The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica, 67(6), 361–370. pmid:6880820
- 27. Lipsy R. J. (2003). The National Cholesterol Education Program Adult Treatment Panel III Guidelines. Journal of Managed Care Pharmacy, 9(1 Supp A), 2–5. pmid:14613351
- 28. Tadel F., Baillet S., Mosher J. C., Pantazis D., & Leahy R. M. (2011). Brainstorm: A user-friendly application for MEG/EEG analysis. Computational Intelligence and Neuroscience, 2011. pmid:21584256
- 29. Luck S. J. (2014). Basic principles of ERP recording. An Lntroduction to the Event-Related Potential Technique, 147–183.
- 30. Melara R. D., Wang H., Vu K. P. L., & Proctor R. W. (2008). Attentional origins of the Simon effect: Behavioral and electrophysiological evidence. Brain Research, 1215, 147–159. pmid:18474363
- 31.
Polich J. (2012). Neuropsychology of P300, in the Oxford Handbook of Event-Related Potential Components (Kappenman E. S. & Steven J. L., Eds.), Oxford University Press, Oxford, UK.
- 32. Folstein J. R., & Van Petten C. (2008). Influence of cognitive control and mismatch on the N2 component of the ERP: A review. Psychophysiology, 45(1), 152–170. pmid:17850238
- 33. Gu L., Chen J., Gao L., Shu H., Wang Z., Liu D., et al. (2018). Cognitive reserve modulates attention processes in healthy elderly and amnestic mild cognitive impairment: An event-related potential study. Clinical Neurophysiology, 129(1), 198–207. pmid:29190525
- 34. Tsolaki A. C., Kosmidou V., Kompatsiaris I. Yiannis , Papadaniil C., Hadjileontiadis L., et al. (2017). Brain source localization of MMN and P300 ERPs in mild cognitive impairment and Alzheimer’s disease: a high-density EEG approach. Neurobiology of Aging, 55, 190–201. pmid:28461101
- 35. Aubert-Broche B., Fonov V.S., García-Lorenzo D., Mouiha A., Guizard N., Coupé P., et al. (2013) A new method for structural volume analysis of longitudinal brain MRI data and its application in studying the growth trajectories of anatomical brain structures in childhood. Neuroimage, 82, 393–402. pmid:23719155
- 36. Coupé P., Yger P., Prima S., Hellier P., Kevrann C., Barillot C. (2008) An optimized blockwise nonlocal means denoising filter for 3-D magnetic resonance images. IEEE Trans Med Imaging, 27(4),425–41. pmid:18390341
- 37. Sled J.G., Zijdenbos A.P., & Evans A.C. (1998) A nonparametric method for automatic correction of intensity nonuniformity in MRI data. IEEE Trans Med Imaging, 17(1),87–97. pmid:9617910
- 38. Eskildsen S.F., Coupé P., Fonov V., Manjón J.V., Leung K.K., Guizard N., et al. (2012) BEaST: Brain extraction based on nonlocal segmentation technique, Neuroimage, 59(3), 2362–73. pmid:21945694
- 39. Collins D.L., Neelin P., Peters T.M., & Evans A.C. (1994) Automatic 3D intersubject registration of MR volumetric data in standardized Talaraich space. J Comput Assist Tomogr, 18(2),192–205. pmid:8126267
- 40. Collins D.L., Holmes C.J., Peters T.M & Evans A.C. (1995) Automatic 3-D model-based neuroanatomical segmentation. Human Brain Mapping, 3(3),190–208.
- 41. Fonov V., Evans A., McKinstry R., Almli C., & Collins D. (2009). Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage, 47, S102.
- 42. Gramfort A., Papadopoulo T., Olivi E., & Clerc M. (2010). OpenMEEG: Opensource software for quasistatic bioelectromagnetics. BioMedical Engineering Online, 9, 1–20. pmid:20051137
- 43. Baillet S., Mosher J. C., & Leahy R. M. (2001). Electromagnetic brain mapping. IEEE Signal Processing Magazine, 18(6), 14–30.
- 44. Chao L. L., Lindgren J. a, Flenniken, D. L., & Weiner, M. W. (2004). ERP evidence of impaired central nervous system function in virally suppressed HIV patients on antiretroviral therapy. Clin Neurophysiol, 115(7), 1583–1591. pmid:15203059
- 45. Polich J. (2000). Neuroelectric assessment of HIV: EEG, ERP, and viral load. International Journal of Psychophysiology, 38(1), 97–108. pmid:11027797
- 46. Tartar J. L., Sheehan C. M., Nash A. J., Starratt C., Puga A., & Widmayer S. (2004). ERPs differ from neurometric tests in assessing HIV-associated cognitive deficit. NeuroReport, 15(10), 1675–1678. pmid:15232305
- 47. Tartar J. L., McIntosh R. C., Rosselli M., Widmayer S. M., & Nash A. J. (2014). HIV-positive females show blunted neurophysiological responses in an emotion-attention dual task paradigm. Clin Neurophysiol, 125, 1164–1173. pmid:24405904
- 48. Gajewski P. D., Ferdinand N. K., Kray J., & Falkenstein M. (2018). Understanding Sources of Adult Age Differences in Task Switching: Evidence from Behavioral and ERP Studies. Neuroscience & Biobehavioral Reviews, 92(June), 255–275.
- 49. Rossini P. M., Rossi S., Babiloni C., & Polich J. (2007). Clinical neurophysiology of aging brain: From normal aging to neurodegeneration. Progress in Neurobiology, 83, 375–400. pmid:17870229
- 50. Bledowski C., Prvulovic D., Hoechstetter K., Scherg M., Wibral M., Goebel R., & Linden D. E. (2004). Localizing P300 generators in visual target and distractor processing: a combined event-related potential and functional magnetic resonance imaging study. Journal of Neuroscience, 24(42), 9353–9360. pmid:15496671
- 51. Walz J. M., Goldman R. I., Carapezza M., Muraskin J., Brown T. R., & Sajda P. (2013). Simultaneous EEG-fMRI Reveals Temporal Evolution of Coupling between Supramodal Cortical Attention Networks and the Brainstem. Journal of Neuroscience, 33(49), 19212–19222. pmid:24305817
- 52. Wang K., Li Q., Zheng Y., Wang H., & Liu X. (2014). Temporal and spectral profiles of stimulus-stimulus and stimulus-response conflict processing. NeuroImage, 89, 280–288. pmid:24315839
- 53. Xia T., Li H., & Wang L. (2016). Implicitly strengthened task-irrelevant stimulus-response associations modulate cognitive control: Evidence from an fMRI study. Human Brain Mapping, 37(2), 756–772. pmid:26595602
- 54. Cacciaglia R., Escera C., Slabu L., Grimm S., Sanjuán A., Ventura-Campos N., & Ávila C. (2015). Involvement of the human midbrain and thalamus in auditory deviance detection. Neuropsychologia, 68, 51–58. pmid:25556848
- 55. Klostermann F., Wahl M., Marzinzik F., Schneider G. H., Kupsch A., & Curio G. (2006). Mental chronometry of target detection: Human thalamus leads cortex. Brain, 129(4), 923–931. pmid:16418179
- 56. Sanford R., Ances B. M., Meyerhoff D. J., Price R. W., Fuchs D., Zetterberg H., et al. (2018). Longitudinal Trajectories of Brain Volume and Cortical Thickness in Treated and Untreated Primary HIV Infection. Clinical Infectious Diseases, l(May), 1–23.
- 57. Sanford R., Fellows L. K., Ances B. M., & Collins D. L. (2018). Association of Brain Structure Changes and Cognitive Function With Combination Antiretroviral Therapy in HIV-Positive Individuals. JAMA Neurology, 75(1), 72. pmid:29131878
- 58. Wade B. S. C., Valcour V. G., Wendelken-Riegelhaupt L., Esmaeili-Firidouni P., Joshi S. H., Gutman B. A., & Thompson P. M. (2015). Mapping abnormal subcortical brain morphometry in an elderly HIV + cohort. NeuroImage: Clinical, 9, 564–573.
- 59. Janssen M. A. M., Hinne M., Janssen R. J., van Gerven M. A., Steens S. C., Góraj B., et al. (2016). Resting-state subcortical functional connectivity in HIV-infected patients on long-term cART. Brain Imaging and Behavior.
- 60. Plessis S. Du, Vink M., Joska J. a, Koutsilieri E, Stein D. J., & Emsley R (2014). HIV infection and the fronto–striatal system. Aids, 28, 803–811. pmid:24300546
- 61. Cole J. H., Caan M. W. A., Underwood J., De Francesco D., van Zoest R. A., Wit F. W. N. M., et al. (2018). No Evidence for Accelerated Aging-Related Brain Pathology in Treated Human Immunodeficiency Virus: Longitudinal Neuroimaging Results From the Comorbidity in Relation to AIDS (COBRA) Project. Clinical Infectious Diseases, 66(June), 1899–1909. pmid:29309532
- 62. Kaur N., Fellows L.K., Brouillette M.J, & Mayo N.E Development and validation of a cognitive reserve index in HIV. J Int Neuropsych Soc, in press. pmid:34085627
- 63. Chan P., Hellmuth J., Spudich S., Valcour V. (2016) Cognitive Impairment and Persistent CNS Injury in Treated HIV, Curr HIV/AIDS Rep,13(4), 209–17 pmid:27188299
- 64. Lam A., Mayo N. E., Scott S., Brouillette M.-J., & Fellows L. K. (2019). HIV-Related Stigma Affects Cognition in Older Men Living with HIV. JAIDS Journal of Acquired Immune Deficiency Syndromes, 80(2), 198–204. pmid:30399037
- 65. Joska J. A., Fincham D. S., Stein D. J., Paul R. H., & Seedat S. (2010). Clinical correlates of HIV-associated neurocognitive disorders in South Africa. AIDS and Behavior, 14(2), 371–378. pmid:19326205