Associating spatial locations with rewards is fundamental to survival in natural environments and requires the integrity of the hippocampus and ventral striatum. In joint multineuron recordings from these areas, hippocampal–striatal ensembles reactivated together during sleep. This process was especially strong in pairs in which the hippocampal cell processed spatial information and ventral striatal firing correlated to reward. Replay was dominated by cell pairs in which the hippocampal “place” cell fired preferentially before the striatal reward-related neuron. Our results suggest a plausible mechanism for consolidating place-reward associations and are consistent with a central tenet of consolidation theory, showing that the hippocampus leads reactivation in a projection area.
Thinking back to an exciting event often includes the scene in which the event took place. Associations between specific places and emotional events are consolidated in memory, but how this is achieved is currently unknown. Two brain areas involved in learning such associations are the hippocampus and the ventral striatum, which represent spatial and emotional information, respectively. A highly valuable object in an environment will prompt humans and animals to take action, such as approaching the object. Here, we demonstrate that a combination of spatial and emotional aspects of a learning experience is replayed in the hippocampus and the ventral striatum during sleep, which is likely to contribute to the consolidation and strengthening of memory traces. This reactivation is coordinated such that the spatial information in the hippocampus is activated shortly before the emotional information in the ventral striatum. This finding is consistent with a central prediction from Memory Consolidation Theory, namely that the hippocampus initiates and orchestrates replay in connected brain areas. In addition, sleep replay occurs at a time scale about ten times faster than during the actual experience, which makes it a mechanism suitable for strengthening synaptic connections associating place with reward. Our results shed new light on the distributed way the brain processes, links, and retrieves different aspects of memories.
Citation: Lansink CS, Goltstein PM, Lankelma JV, McNaughton BL, Pennartz CMA (2009) Hippocampus Leads Ventral Striatum in Replay of Place-Reward Information. PLoS Biol 7(8): e1000173. https://doi.org/10.1371/journal.pbio.1000173
Academic Editor: Charles F. Stevens, Salk Institute for Biological Studies, United States of America
Received: November 17, 2008; Accepted: July 7, 2009; Published: August 18, 2009
Copyright: © 2009 Lansink et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by Human Frontier Science Program grant RGP0127/2001, NWO-VICI grant 918.46.609, Senter Novem BSIK grant 03053 and EU grant FP7, #217148 to CMAP and grant MH046823 from the National Institute of Mental Health to BLM. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: EV, explained variance; HC, hippocampus; LFP, local field potential; QW-SWS, quiet wakefulness–slow-wave sleep; REM, rapid eye movement; REV, reversed explained variance; SWS, slow-wave sleep; VS, ventral striatum
Successful foraging requires that animals maintain a representation of a multitude of reward properties including the location at which a reward can be found. Forming a place–reward association is thought to depend critically on the communication between the hippocampal formation and the ventral striatum (VS). Cells in the hippocampus proper (HC) , and adjacent subiculum  show location-specific firing (i.e., “place fields”), and these structures are crucial for spatial and contextual learning ,–. Neurons in the VS fire in relation to rewards, as they are expected or actually delivered, as well as to cues predictive of reward –. Receiving information from a range of structures such as the HC, amygdala, prefrontal cortex, and midline thalamic nuclei –, the VS is thought to utilize information of reward-predicting cues and contexts to guide goal-directed behavior ,,. This process is under strong control of the mesolimbic dopaminergic system, and its disruption has been associated with neuropsychiatric conditions such as drug addiction and obsessive-compulsive disorder –. Although the hippocampal formation projects directly to the VS, and this connection has been implicated in contextual conditioning , it is unknown how neural representations of contextual and motivational information are integrated and stored to enable the learning of place-reward associations.
In several brain areas, neuronal patterns evoked during behavior are reactivated during subsequent sleep –. Through modification of synaptic connections, this reactivation has been theorized to constitute an important step in memory consolidation –. Because hippocampal CA1 pyramidal cells exhibiting place fields during active behavior have been demonstrated to reactivate during sleep, it may be reasonably assumed that this replay pertains to spatial and contextual information ,,,. In contrast, reactivation in the VS is dominated by reward-related information . Joint reactivation of HC and VS may enable the formation of a memory trace comprising both contextual and motivational components. In this study, we recorded activity from neuronal ensembles in the rat HC and VS simultaneously during wake and sleep episodes to examine whether the HC and VS reactivate coherently and to reveal the temporal dynamics of this process. First, during active behavior, much of the dynamics of hippocampal processing is governed by the theta rhythm, which has been hypothesized to function as a “read-in” or encoding mode for information acquisition and provides a means to temporally align spike sequences by way of theta phase precession ,–. Therefore, we studied whether neural activity modulation by this rhythm in the awake state is correlated to reactivation during sleep. A second foremost question in this field, not yet addressed in previous multi-area recording studies ,,, is whether cross-structural replay depends on the type of behavioral information coded by cell assemblies. To address this question, we investigated whether reactivation is preferentially associated with the expression of place fields and reward-related neural responses. Third, we planned to utilize joint HC-VS recordings to test a central tenet of theories of memory consolidation –. These theories posit that, after a learning experience, long-term episodic and declarative memories become gradually independent of hippocampal storage because this structure would repeatedly retrieve stored associative information over time and thereby orchestrate consolidation of memory traces in the neocortex and other target sites. A key point in these hypotheses is that replay is initiated and orchestrated by the HC, which prompted us to examine whether hippocampal activity leads the VS during reactivation.
Four rats were implanted with a tetrode drive allowing joint HC-VS recordings of spike trains of multiple neurons and local field potentials (LFPs) in each area. Daily recording sessions were composed of an episode of reward searching behavior flanked by two episodes of rest, which rats spent on a “nest” next to the track. The task was to run along a triangular track repeatedly and in one direction. On each lap, one of three reward wells was baited with a drop of one of three corresponding reward types; i.e., sucrose solution, vanilla desert, or chocolate mousse. An example of joint HC and VS ensemble recordings during track running and sleep is shown in Figure 1. First, we assessed reactivation of neuronal patterns using an explained variance (EV) method based on the spike correlations of cell pairs across all simultaneously recorded neurons ,. The EV reflects the extent to which the variance in the distribution of spike correlations during postbehavioral rest is statistically accounted for by the correlation pattern found during track running, factoring out the correlations present in prebehavioral rest. Joint HC-VS reactivation was examined during rest periods in which the rat was immobile, using only spike correlations between pairs composed of one HC and one VS neuron.
Example of the firing patterns of concurrently recorded hippocampal (HC1–HC4) and ventral striatal (VS1–VS3) cells during track running (A) and QW-SWS (B). Only cells that exhibited a place field or a reward-related correlate are shown in these graphs. Local field potentials recorded near the hippocampal fissure (A) and the hippocampal pyramidal cell layer (B) are plotted in parallel. (A) When the rat ran along the triangular track, the LFP displayed an oscillation of theta frequency (6–10 Hz, top: raw trace; bottom: filtered trace [6–10 Hz]). During the plotted period of 25 s, the rat encountered six reward sites (s = sucrose solution, v = vanilla desert, c = chocolate mousse) of which three contained a reward (green arrow) and the others were empty (red arrows). Each row in the black field represents one cell; its spikes are shown with colored dots. (B) During QW-SWS, the LFP displays large irregular activity intermitted with sharp wave-ripple complexes (top: raw trace; bottom: filtered trace [100–250 Hz]). Identified ripples are indicated with an asterisk (*). Several units that were activated during track running were reactivated within a short time period. Note the different time scales in (A and B). The relative firing order of pairs of HC and VS cells roughly corresponds to that observed during behavior (A).
We found coherent, cross-regional reactivation between ensembles of the HC and VS as expressed by an EV of 9.7±3.0%, which was significantly higher than the control measure, the reverse explained variance (REV: 1.4±0.5%, p<0.01, n = 21 sessions; Figure 2A and 2B; Figures S1 and S2; Table S1; Text S1). In the analyses conducted in this research, putative interneurons were excluded from the neuronal population, but it should be noted that including these interneurons yielded similar reactivation values (EV: 9.4±2.7%, REV: 1.9±0.7%; p<0.01). Analysis of the temporal dynamics of reactivation in 20-min blocks of concatenated rest revealed a gradually decaying reactivation which was significant for at least 1 h of postbehavioral rest (Figure 2C). Within periods of rest, reactivation was prominent especially during quiet wakefulness–slow-wave sleep episodes (QW-SWS; n = 13 sessions), but it was not significant for rapid eye movement (REM) sleep (Figure 2D). The lack of pattern recurrence during REM sleep was not attributable to its relatively short duration, undersampling of spikes, or its late occurrence after sleep onset compared to QW-SWS (Figure S3; Table S2; Text S1). Reactivation in the HC  and VS  occurs markedly during sharp wave-ripple complexes, i.e., short-lasting, high-frequency oscillations in the hippocampal LFP with associated bursts of large-scale neuronal firing that characterize QW-SWS ,. The same trend was observed for joint reactivation; however, the difference in reactivation values for time windows of 200 ms following ripple onset (“Ripples”) and during 200 ms following the onset of interripple intervals (“Intervals”) did not reach statistical significance, which most likely relates to the high variability across sessions (cf. ) (Ripples: EV: 5.9±2.6%, REV: 1.0±0.3%; Intervals: EV: 1.9±0.8%, REV: 0.5±0.2%; EV and between Ripples and Intervals [EV−REV]: n.s.; n = 14 sessions).
(A) Diagrams representing firing pattern correlations for pairs of simultaneously recorded hippocampal and ventral striatal units during periods of active behavior and rest in a single session. Individual neurons are represented as dots around the perimeter of a circle (filled dots: hippocampal CA1 units, n = 10; open dots: ventral striatal units, n = 13). Lines indicate a significant firing correlation between two neurons (red: positive, yellow: negative correlations). A pattern of correlations emerges during track running and is reinstated in postbehavioral rest, whereas it was largely absent in rest preceding behavior. (B) The EV was significantly larger than the control value (REV) when compared across sessions (**p<0.01). Error bars represent the standard error of the mean (SEM). (C) Temporal dynamics of joint reactivation were examined in three 20-min blocks of rest. Reactivation occurs at least up to an hour of rest after the experience but decays gradually (***p<0.002). (D) Reactivation observed during QW-SWS (**p<0.01) was not different from that found for the entire rest episodes (Rest, *p<0.05; n = 13 sessions). Reactivation was not observed in REM sleep. Between-condition statistics hold for EV values and the difference between [EV−REV].
The existence of joint HC-VS reactivation raises the question of which physiological and behavioral factors are associated with the strength of this process. We examined three not mutually exclusive factors pertaining to (1) the modulation of the neural activity patterns by theta oscillations, (2) the correlation of neuronal firing patterns with behavioral parameters, and (3) the order in which neurons in different areas were activated. First, we computed the degree to which cells in each pair fired together, and then all of these correlation values per episode were pooled across sessions and animals. We next formed subgroups of cell pairs by partitioning the complete set of correlation values on the basis of the factor under scrutiny. Reactivation values were computed for these subgroups, and statistical significance was assessed by applying a bootstrapping procedure with resampling of pooled correlation values ,.
The two-stage model of memory trace formation posits that theta oscillations are crucial for encoding information in the HC in the awake, active state . The hippocampal theta rhythm may also have a role in governing the temporal organization of activity in target structures to ensure efficient communication ,. Thus, our first hypothesis holds that HC-VS reactivation will only be strong when information is cross-structurally aligned during encoding by a common temporal framing, the theta oscillation, creating windows of near-synchronous firing.
During track running, robust theta oscillations were observed in the HC and VS. In both areas, cells were observed whose firing patterns were modulated by the local theta rhythm (n = 121 out of 263, 46.0%, in HC, and n = 20 out of 243, 8.2%, in VS (Figure 3A; FigureS4A). Ventral striatal units that were modulated by the local theta rhythm generally showed firing rate modulation also by hippocampal theta oscillations (n = 20, 8.2%). When the peak of the theta oscillation recorded near the hippocampal fissure was taken as synchronizing time point, CA1 cells fired at an average angle of 199.9±6.1° (range: 43.9°–356.4°) and VS cells fired at a slightly later phase (217.1±26.0°, range: 4.5°–336.3°; n.s.). Cell pairs were divided on the basis of modulation by the hippocampal theta rhythm, resulting in four subgroups: Both Cells (n = 140), HC only (n = 1,273), VS only (n = 81), and None Modulated (n = 1,422). Reactivation was observed for all but the VS only groups; its strength was significantly stronger in the Both Cells group compared to the None Modulated and HC only groups (Figure 3A; Table S3).
(A) Modulation by theta oscillations. Four examples show binned spike counts (upper panels, solid lines) in relation to the hippocampal theta rhythm (bottom panels) for two successive theta cycles. Randomization of spike intervals abolished the relation between firing pattern and theta rhythm (dashed lines). Distributions of EV and REV values for each subgroup, obtained with bootstrapping, showed significant reactivation in all but the VS only subgroups. Reactivation in the Both Cells group was significantly stronger than in the other groups (p<1×104; right-hand panel). (B) Processing of place and reward information. The left panel shows the spatial distribution of local firing rates of two hippocampal–ventral striatal pairs. The rat's trajectory is shown in black, and the firing rate of the neurons is color coded, ranging from low rates in dark red to their individual maxima (top right corners) in yellow and white. The Double Correlates group reactivated significantly, whereas the other subgroups did not. PF: place field; RRU, reward-related unit. (C) Firing order was defined by the difference in area between the light- and dark-shaded regions of cross-correlograms. Reactivation was observed in all groups, but the HC→VS group reactivated more strongly than the other groups.
Our second hypothesis departed from the assumption that spike patterns are not reactivated equally, but are reprocessed especially when they convey behaviorally relevant information. For the HC-VS system, we predicted that cells expressing spatial (HC) and reward-related information (VS) should be preferentially reactivated. Location-specific firing was found for 102 out of 263 (38.8%) hippocampal cells. Place fields were distributed uniformly across the track; there was no indication that place fields occurred more frequently near reward sites or corners of the track. In contrast, a subset of VS neurons fired in close temporal relationship with reward site visits (41 out of 243 cells, 16.9%; Text S1). Reward-related responses were generally increments in firing rate and could be generated at one, two, or all three reward sites. Furthermore, they were often sensitive to either the presence or absence of reward. In line with previous studies on the VS ,,, we will apply the term reward-related to all VS units showing significant responses time-locked to reward site visits.
Depending on the expression of place fields and reward-related correlates, cell pairs were grouped in four categories: Double Correlates (n = 192), Place Field only (n = 941), Reward-related Correlate only (n = 287), or No Correlates (n = 1496). The Double Correlates group showed very strong reactivation (EV: 22.9%, REV: 0.1%), whereas reactivation in the other three subgroups in this partition was not significant (Figure 3B; Table S3). Accordingly, the strength of coactivation of a place cell and a reward-related VS cell, expressed in the Pearson correlation coefficient, was positively correlated to the degree of spatial overlap of the firing fields on the track during task performance and postbehavioral sleep, but not during prebehavioral rest (prebehavioral rest: n.s.; track running: R2 = 0.25, p<1×10−12, postbehavioral rest: R2 = 0.03, p<0.02; n = 192).
A long-standing assumption in memory consolidation theory holds that the HC initiates and orchestrates reactivation in its projection areas –. This general process may be realized in several ways (see Text S1). In the HC-VS system, evidence suggests a particular variant of replay in which hippocampal ripples initiate reactivation locally and subsequently trigger a wave of enhanced excitability in the VS . This variant implies that reactivation should be strong when a particular firing order is maintained: during replay, a hippocampal cell should fire predominantly in advance of a VS cell. During behavior, VS firing may also precede HC firing, but this order should not be associated with strong reactivation. The HC→VS order would also be consistent with the unidirectionality of the projection from HC to VS . Despite the finding that sleep reactivation occurs in a “forward” direction, meaning that the order of firing during sleep is similar to the order during the preceding behavior ,,, this critical assumption has yet to be confirmed or refuted. Hence, our third hypothesis holds that reactivation is strong when the information flow is organized according to a leading role of the HC.
The firing order of each cell pair was assessed by computing cross-correlograms , and determining which order of firing was most prevalent using a “temporal bias” measure . Three subgroups were distinguished, i.e., HC→VS pairs (n = 608), VS→HC pairs (n = 796), and No Clear Order, which included pairs that did not show a preferred firing order (n = 1,512). The HC→VS group reactivated strongly (EV: 15.2%, REV: 0.0%). Reactivation was also observed for the other groups, although the observed strengths were significantly lower than for the HC→VS group. (Figure 3C; Table S3).
Variations in reactivation measured across all of the subgroups partitioned according to each of the three factors could not be attributed to differences in varying numbers of cell pairs, differences in correlation strengths, or differences in spike counts (Text S1). Altogether, these results suggest that all three factors analyzed—modulation of both cells by the hippocampal theta rhythm, maintenance of the HC→VS firing order, and expression of a combination of a place field and a reward correlate—are associated with strong reactivation. However, since a reactivating cell pair may display multiple characteristics at the same time (e.g., behavioral correlates and a particular firing order; see Figure S4), we used a multilinear regression model to test whether the contribution of each cell pair to the session EV value was dependent on firing order, theta modulation, behavioral correlates, or any combination of these characteristics. First, the relative contribution of each cell pair to the session reactivation was estimated by excluding a pair from the simultaneously recorded population and recomputing the reactivation values. The difference between the session EV minus the EV after pair exclusion represents the estimated contribution of that pair to the session EV. Multilinear regression showed that both the expression of a double correlate and the HC→VS firing order were significant factors in explaining the contribution to the session EV (p<0.02 and p<0.002, respectively; theta modulation was not significant, p = 0.6). The combination of firing order and expression of a double correlate predicted the pair's contribution better than either one alone (p<0.0002). This analysis confirms the importance of the HC→VS firing order and expression of combined place and reward information during track running for subsequent reactivation and identified theta modulation as a less significant indicator.
We tested whether reactivation in the subgroup reactivating most strongly, i.e., the Double Correlates, was sparsely distributed as we previously showed for VS ensembles . First, we assessed the contribution of each cell pair to the reactivation as explained above. To find an indication of how many cell pairs can be excluded to abolish reactivation, the pairs were sorted in descending order in terms of their contribution to the reactivation value [EV−REV]. Then the pairs were excluded one by one in a cumulative fashion starting with the highest contributor from the population, and reactivation values were computed each time a next pair was excluded. If the 17 (17/192, 8.9%) most contributing cell pairs were left out of the population, the [EV−REV] dropped below 5.0%. A total of 34 (17.7%) pairs could be removed before the [EV−REV] level decreased to 0.0%. This analysis indicates that, consistent with VS ensembles, reactivating cell pairs were also sparsely distributed in the HC-VS population.
We next explored whether HC-VS cell pairs fire in the same order during reactivation as during behavior and whether replay is accelerated relative to active behavior. For each pair of neurons that showed a place field and a reward-related correlate, we constructed three cross-correlograms, one for each task-episode (n = 192, Figure 4A). We compared the time offsets during active behavior and rest for pairs that showed significant peaks in the cross-correlograms of track running and in at least one of the rest episodes (n = 53, 27.6%). The time offsets of the peaks during track running were positively correlated to those in postbehavioral rest (R2 = 0.09, p<0.05; n = 47), but not to those of prebehavioral rest (Figure 4B; n = 26). Thus, the recurrent firing patterns reflected the preceding experience. In this analysis, spatial overlap between the firing fields of a cell pair turned out not to be a prerequisite for concurrent firing during subsequent sleep, as 29.8% of the cell pairs that showed peaks in the cross-correlograms for task performance and postbehavioral rest exhibited nonoverlapping firing fields on the track. The peak offset in the cross-correlograms of track running ranged from −4.5 to 3.8 s and was significantly correlated to the spatial distance between the firing fields (R2 = 0.27, p<0.001).
(A) Cross-correlograms for three pairs of simultaneously recorded neurons showing the temporal relation of firing during prebehavioral rest, track running, and postbehavioral rest. Hippocampal activity is synchronized on ventral striatal firing (time = 0, bin size 20 ms). Spatially distributed firing patterns of the neurons are shown in the blue squares (left: HC, bottom: VS; see also Figure 3B). During track running, the three pairs of neurons show correlated firing with peaks at different offsets relative to time zero. This correlated firing was absent in prebehavioral rest but reinstated during postbehavioral rest (indicated by arrows). In the topmost example, a secondary peak during track running is recurring in postbehavioral rest (indicated by the asterisk [*]). The relative time offsets of peaks were preserved from track running to postbehavioral rest. (B) Scatter plots of the temporal offsets of the peaks in the cross-correlograms (CCG) during track running and prebehavioral rest (left) and postbehavioral rest (right). The peak offsets during track running were correlated to the peak-offsets during postbehavioral rest (R2 = 0.09, p<0.05), but not to prebehavioral rest. The peak offsets during postbehavioral rest were significantly reduced compared to track running, indicating accelerated reactivation. Note that cell pairs showing a significant peak in the cross-correlograms of track running and of at least one rest episode were included in analysis. Therefore, the number of data points is different in the left and the right panels.
To determine whether the order of firing on the track was preserved or reversed in the subsequent rest episode the offset sign (+ or −) of the cross-correlogram peak relative to zero was considered. Peaks during track running and postbehavioral rest were consistently found with the same offset sign (43/47 = 89%, sign test, p<0.0001), which demonstrates that replay took place in a forward direction. In combination with the strong reactivation of cell pairs that exhibited the HC→VS firing order during track running observed in the subgroup-based reactivation analysis (Figure 3C), the preservation of firing order suggests that replay should be dominated by activity patterns in which HC firing largely precedes VS firing, both during track running and postbehavioral rest. Indeed, in the large majority of cell pairs that showed forward reactivation, the hippocampal cell fired preferentially before the striatal cell during both periods (36/43 = 83.7%), indicating that most of the reactivating cell pairs express a HC→VS order during behavior and sleep (sign test, p<0.0001). Thus, not only is the firing order preserved from the behavior to ensuing sleep, but apparently the HC also takes the lead in replay and the VS follows.
An additional analysis on all cell pairs with significant cross-correlogram peaks yielded similar results and confirmed that the preferential firing order during reactivation was not attributable to a lack of VS→HC correlations during track running (see Text S1). Like cross-structural reactivation, reactivation within hippocampal and ventral striatal ensembles also took place in a forward direction (see Text S1) (cf.  for HC).
Replay may occur at a different time scale than applicable during behavior ,,. We examined whether joint HC-VS firing patterns were replayed on an accelerated time scale. Peak times in the postbehavioral rest cross-correlogram occurred significantly closer to zero than during track running (track: −525.5±201.9 ms, postbehavioral rest: −53.2±28.5 ms; p<0.01; n = 47), showing an approximately10-fold compression (Figure 4). Replayed patterns appeared compressed and not merely truncated, because the shape of the cross-correlogram peaks with offsets of up to several seconds during behavior were re-expressed during sleep, including their offset from zero (Figure 4A).
Altogether, our results demonstrate coherent reactivation between the HC and a subcortical structure, and identify two major factors governing cross-structural reactivation in the HC-VS system, suggesting a plausible mechanism for consolidation of associative place-reward information. The first factor that significantly correlated to strong HC-VS reactivation bears on the dependence of reactivation on the coding of behaviorally relevant information. Cell pairs that exhibited a double correlate—one place field plus one reward-related correlate—showed the strongest reactivation among all four subgroups. In addition, the contribution to reactivation by individual pairs depended specifically on such a coexpression of behavioral correlates. The near-synchronous reiteration of spatial and motivational information during sleep may serve to integrate these types of information and support the learning of place-reward associations. Such associations are essential to predict and localize desired food and liquids within a known environment and are therefore fundamental to foraging behavior and learned behaviors such as conditioned place preference ,,. Like intra-area ventral striatal reactivation , cross-structural replay is dependent on a relatively small subset of cell pairs, indicating that it is a sparsely distributed phenomenon. If replay indeed supports memory consolidation, the formation of associations of a specific place-reward combination is likely to depend on a small minority of cells in the HC-VS circuitry.
The second significant factor in joint HC-VS replay is the preferred firing order of HC and VS cells, consistent across track running and subsequent sleep. The HC→VS firing order during track running was associated with a significantly elevated reactivation as compared to other temporal relationships, and the cell pair contributions to reactivation depended on this specific firing order. This organization of firing order obeys the direction of the anatomical projection , and presents necessary, although not sufficient, evidence for a central tenet of consolidation theory, proposing the HC to initiate reactivation in its target structures, as predicted by Marr  and subsequent theorists – .
Our data provide several indications supporting that the observed cross-structural reactivation is the consequence of a coordinated process between the HC and the VS rather than of two separately, or coincidentally, reactivating ensembles. First, the temporal relationship between a pair of task-related hippocampal and ventral striatal cells was relatively consistent across task phases, resulting in significant peaks in the cross-correlograms of a substantial number of pairs both during track running and postbehavioral rest. If joint reactivation was just coincidental, the temporal firing relationship between cells in different structures is expected to be random, contrary to what was observed. Second, the timing of the peaks during track running and postbehavioral rest was correlated in the cross-correlogram analysis (Figure 4B), and furthermore, the time scale of sequential activation of firing patterns during postbehavioral rest was compressed compared to track running. Thus, temporal firing relations were consistent across different overall brain states (awake active versus SWS) and on accelerated time scales. Observing such results is very unlikely if the two structures would be reactivating without a systematic temporal relationship. Third, during behavior we identified pairs that fired in the order HC→VS and also in the order VS→HC. During reactivation in the postbehavioral rest, however, we found an overrepresentation of HC→VS pairs. If two ensembles reactivated independently, one would expect the ratio of HC→VS and VS→HC pairs to be similar during behavior and reactivation.
An important finding is that the joint reactivation is compressed by a factor of ten compared to the behavioral time scale of neuronal activation. Thus, at least several seconds of “real-time” joint place-reward information during behavior are brought together in a time frame of hundreds of milliseconds during sleep. This further supports the plausibility of a mechanism for the associative storage of place and reward information by way of synaptic weight changes in the HC-VS system. If a cell from the hippocampal formation, coding place, fires consistently and briefly in advance of a VS cell signaling reward (Figure 4), spike timing–dependent plasticity may be induced in their connection ,.
Cross-correlogram analysis revealed that joint reactivation is not restricted to neuronal pairs that exhibit overlapping firing fields; peaks that were separated by up to about 4.5 s during behavior were found to recur during postbehavioral rest. In a scenario in which a series of place fields is followed by a reward-related correlate, this indicates that value information during SWS is not only paired with locations nearby, but also with more remote stages of the path leading to the reward site. Formation of reward-predicting representations should, by definition, obey the temporal order of predictor-reward events, a requirement that is met by the preferential HC→VS firing order during replay. In principle then, the characteristics of hippocampal-striatal replay are suitable for mediating the “backwards” association between reinforcements and cues and contexts situated progressively earlier in time. This temporally backwards referral is a key feature of conditioning theory and models of reinforcement learning –.
Although the causal role of ensemble reactivation in memory consolidation remains to be proven, the temporally ordered cross-structural replay of spatial and motivational information during sleep illuminates a plausible offline mechanism by which information processed in different parts of the brain can be integrated to enable the composition and strengthening of memory traces comprising various attributes of a single learning experience.
Materials and Methods
All experimental procedures were in accordance with Dutch national guidelines on animal experimentation.
Four male Wistar rats (375–425 g; Harlan) were individually housed under a 12/12-h alternating light-dark cycle with light onset at 8:00 am. All experiments were conducted in the animal's inactive period. During training and recording periods, rats had access to water during a 2-h period following the experimental session, whereas food was available ad libitum. Rats were chronically implanted with a microdrive  containing five individually movable tetrodes directed to the dorsal hippocampal CA1 area (4.0 mm posterior and 2.5 mm lateral to bregma) and seven to the VS (1.8 mm anterior and 1.4 mm lateral to bregma) . Reference electrodes were placed in the corpus callosum dorsal to the HC, and near the hippocampal fissure. A skull screw inserted in the caudal part of the parietal skull bone served as ground.
Unit activity, local field potentials, and position data were acquired on a 64-channel Cheetah recording system (Neuralynx). Spike sorting was performed offline using custom cluster-cutting software as described in Text S1.
Recordings of hippocampal CA1 neurons were made from 103 locations between 2.6 mm and 4.8 mm posterior and between 1.2 mm to 2.8 mm lateral to bregma compared to an atlas of the rat brain . Ventral striatal tetrodes were situated between approximately 2.2 and 1.2 mm anterior to bregma and between 1.6 and 3.0 mm laterally. From a total of 140 recording sites, 58% was estimated to be situated in the core region and 42% in the shell region of the VS. Although most sessions were likely to contain recordings from both the core and shell region, six sessions were identified to contain core-only recordings. No gross differences were observed in the number, firing rate, or appearance of behavioral correlates that were estimated to be recorded from the core and the shell region. Moreover, cross-regional reactivation was observed for the core-only sessions, with EV and REV values similar to those observed for other sessions. Therefore, core and shell recordings were pooled.
Resting State and Sleep Phase Identification
Pre- and postbehavioral rest episodes included all periods of at least 20 s in which the rat was in the flower pot and remained motionless; i.e., episodes of movement were excluded from analysis. Within these periods of rest, episodes of SWS were characterized by the presence of large irregular activity and the occurrence of sharp wave-ripple complexes in the LFP of the CA1 pyramidal layer ,. Ripples were detected each time the squared amplitude of the filtered LFP trace (100–300 Hz) crossed a threshold of 3.5 standard deviations (SD) for at least 25 ms. Because incidentally short periods of quiet wakefulness may have been included in SWS episodes, as these two phases share principal LFP characteristics, this state is referred to as quiet wakefulness–slow-wave sleep (QW-SWS). REM sleep periods were indicated by an elevated ratio (>0.4) of spectral density in the theta band (6–10 Hz) to the overall power of the LFP trace recorded near the hippocampal fissure. Their borders were refined upon visual inspection of the trace.
Quantification of Reactivation
Session-based reactivation analysis.
The assessment of covariation in firing rates and reactivation within an experimental session with the EV method was previously described ,,,. The temporal correlation between the firing patterns of two neurons was expressed in a Pearson's correlation coefficient that was computed for all concurrently recorded cell pairs using binned spike trains (50-ms bin size) of each rest/active episode. For assessment of cross-structural reactivation, pairs always consisted of one hippocampal and one ventral striatal cell; i.e., intra-area pairs were not taken into account. Separate analyses were conducted to examine intra-area reactivation; see Figure S2 and Text S1. All Pearson correlation coefficients of a particular episode (i.e., prebehavioral rest, track running, and postbehavioral rest) were assembled into a single matrix, and the similarity between the matrices was determined by computing a correlation coefficient for each of the three possible combinations of rest/active episodes. These matrix-based correlation values were used to assess which proportion of the variance in the postbehavioral correlation pattern can be explained by the pattern established during track running while controlling for any correlations that were present before the track running experience (i.e., in the prebehavioral rest). This quantity is expressed in the EV measure:in which R1 and R2 represent the pre- and postbehavioral rest episodes, respectively, and for example, rTrack,R2 equals the matrix correlation coefficient between the Track and Rest 2 pattern. EV equals the square of the partial correlation coefficient and is bounded between 0 and 1. The within-session control measure, i.e., REV, was derived by exchanging R1 and R2 in the previous equation, thereby switching the temporal order of episodes. EV and REV values were computed for all sessions that contained at least five well-isolated active neurons from each area and for 20-min time blocks of concatenated periods of quiet rest and sleep, i.e., periods of active behavior were excluded. Sessions that showed reactivation (EV>REV) in the first 20-min rest block after track running were used to assess decay of reactivation (n = 15). Sleep phase–dependent reactivation was computed for sessions that showed more than 4 min of REM and QW-SWS in each rest episode (n = 13). Differences between EV and REV values were tested for statistical significance with the Wilcoxon matched-pairs signed rank test.
Reactivation analysis for subgroups.
For assessing reactivation in subgroups of cell pairs (e.g., “Both” and “None” modulated in the section on theta modulation), all Pearson correlation coefficients for cell pairs within each subgroup were pooled across all sessions and all rats before the EV and REV values were computed. Estimates of the mean and variance of these values were derived using a bootstrapping procedure in which random samples were taken (n = 10,000) from the observed set of Pearson correlation coefficients ,. During resampling, the triplets of correlation coefficients belonging to the three rest/active episodes were kept together, and the created samples were of the same size as the original subset. The resampling procedure included replacement, thus samples may have contained multiple copies of one triplet, whereas others were omitted. For each sample, reactivation measures were computed resulting in distributions of EV and REV and the difference [EV−REV] values for each subset. After bootstrapping, 95% confidence intervals were determined for the [EV−REV] distribution. A subgroup was considered to reactivate when the 2.5 percentile of this distribution exceeded zero. Differences in reactivation strength between two subgroups were statistically assessed using the [EV−REV] values in Mann-Whitney U test (MWU; p<1×10−4).
Modulation of a cell's firing pattern to the theta oscillation was determined by first filtering LFP traces recorded from the hippocampal fissure and the VS using a Chebyshev type-1 bandpass filter between 6 and 10 Hz. Binned spikes (10°/bin) were then plotted relative to the theta peaks of two successive theta periods. The spike distribution was considered nonuniform when the Rayleigh score was <1×10−5. The phase angle of the spikes was determined by computing the Hilbert transform of the filtered theta signal. Firing of a unit was considered as being modulated by the theta rhythm when shuffling of the spikes abolished the nonuniformity of the spike firing distribution as assessed with the Rayleigh score.
Identification of Place Fields
To characterize spatially selective firing fields, instantaneous firing rates were computed for bins of 50 ms. The spatial position of the rat's head was determined by creating a one-dimensional representation of the track and using a resolution of 2.3 cm. Mutual information was computed between the binned spike trains and the position, and corrected for finite sampling size ,. A cell was considered to express a place field if its firing rate during track running was at least 0.3 Hz and if it carried at least 0.25 bits/spike of spatial information.
Identification of Reward-Related Firing Patterns
Peri-event time histograms were constructed for the rewarded and nonrewarded condition for each reward site and were synchronized on crossings of offline installed “virtual photobeams” positioned at the points where the rat was just reaching the reward sites. Reward-related responses were assessed within a period of 1 s before and 1 s after arrival at a reward site, using a bin resolution of 250 ms. Spike counts in the eight bins comprising the reward period were each compared to three separate control bins taken from the corner passage opposite to the well under scrutiny within the same lap. A bin of the reward period was only considered significantly different when the Wilcoxon matched-pairs signed rank test indicated significance from each of the three control bins (p<0.01). A reward-related response comprised one or more bins that were significantly different from control bins. Control bins did not show marked deviations from overall baseline firing as checked in peri-event time histograms and plots of the spatial distribution of firing rates. Differences between the responses at different reward sites were assessed with a Kruskal-Wallis test (p<0.05) followed by a MWU (p<0.05), whereas rewarded versus nonrewarded conditions were compared using MWU (p<0.05).
Cross-Correlograms and Temporal Bias Method
Cross-correlograms were constructed according to Perkel et al.  and Eggermont . Spikes were binned into 10- or 50-ms intervals, and the cross-correlation was examined across at least three time windows; viz. [−500, 500] ms, [−2,000, 2,000] ms and [−5,000, 5,000] ms. The firing order of pairs of hippocampal and striatal cells was assessed with the temporal bias method  using cross-correlograms with the spikes of the striatal cell serving as reference. The ordinate expressed spike counts per second, which was integrated across intervals of 200 ms before (I) and after (II) zero. The difference between II minus I divided by the sum of the spike counts determined the temporal bias score. If this score was negative, the HC was determined to fire preferentially before the striatal cell. If this score was positive, the preferred firing order was in the opposite direction. The classification No Clear Order was assigned when the scores (I and II) were approximately equal or when the cross-correlogram did not have a clear single maximum.
To estimate the significance of peaks in the cross-correlograms, the mean expected number of joint spike counts μ and the levels of μ±3 SD (corresponding to p = 0.0013) were computed to provide indications for nonrandom excursions of spike counts above or below the expected range . Each cross-correlogram was then subjected five times to a spike-shuffling subtraction procedure ,. Peaks were accepted as significant only when they exceeded the +3 SD threshold above the mean in each of the five repetitions.
Cross-structural reactivation is cell and time specific.
(0.05 MB PDF)
Cross-structural and intra-area reactivation.
(0.04 MB PDF)
Lack of REM sleep reactivation is not attributable to undersampling or decay.
(0.05 MB PDF)
Distributions of firing properties during track running across hippocampal and ventral striatal units.
(0.35 MB PDF)
Numbers of sessions, cells, and cell pairs recorded per rat.
(0.02 MB PDF)
Mean spike counts per cell during REM sleep and QW-SWS segments.
(0.02 MB PDF)
Reactivation values of subgroups composed on the basis of three physiological and behavioral factors.
(0.07 MB PDF)
We thank Ruud Joosten for help with construction of the tetrode arrays and with surgeries, Jolanda Verheul for help with data analysis, and our colleagues at the technology departments of the University of Amsterdam and the Netherlands Institute for Neurosciences for technical assistance.
The author(s) have made the following declarations about their contributions: Conceived and designed the experiments: CSL BLM CMP. Performed the experiments: CSL PMG. Analyzed the data: CSL PMG. Contributed reagents/materials/analysis tools: CSL PMG JVL. Wrote the paper: CSL CMP.
- 1. O'Keefe J, Dostrovsky J (1971) The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res 34: 171–175.
- 2. O'Keefe J, Nadel L (1978) The hippocampus as a cognitive map. Oxford (United Kingdom): Clarendon Press. 570 p.
- 3. Barnes C. A, McNaughton B. L, Mizumori S. J, Leonard B. W, Lin L. H (1990) Comparison of spatial and temporal characteristics of neuronal activity in sequential stages of hippocampal processing. Prog Brain Res 83: 287–300.
- 4. Morris R. G, Garrud P, Rawlins J. N, O'Keefe J (1982) Place navigation impaired in rats with hippocampal lesions. Nature 297: 681–683.
- 5. Kim J. J, Fanselow M. S (1992) Modality-specific retrograde amnesia of fear. Science 256: 675–677.
- 6. McNaughton B. L, Barnes C. A, Gerrard J. L, Gothard K, Jung M. W, et al. (1996) Deciphering the hippocampal polyglot: the hippocampus as a path integration system. J Exp Biol 199: 173–185.
- 7. Apicella P, Ljungberg T, Scarnati E, Schultz W (1991) Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85: 491–500.
- 8. Schultz W, Apicella P, Scarnati E, Ljungberg T (1992) Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610.
- 9. Roitman M. F, Wheeler R. A, Carelli R. M (2005) Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45: 587–597.
- 10. Kelley A. E, Domesick V. B, Nauta W. J (1982) The amygdalostriatal projection in the rat–an anatomical study by anterograde and retrograde tracing methods. Neuroscience 7: 615–630.
- 11. Berendse H. W, Galis-de Graaf Y, Groenewegen H. J (1992) Topographical organization and relationship with ventral striatal compartments of prefrontal corticostriatal projections in the rat. J Comp Neurol 316: 314–347.
- 12. Berendse H. W, Groenewegen H. J (1990) Organization of the thalamostriatal projections in the rat, with special emphasis on the ventral striatum. J Comp Neurol 299: 187–228.
- 13. Groenewegen H. J, Vermeulen-Van der Zee E, te Kortschot A, Witter M. P (1987) Organization of the projections from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutinin. Neuroscience 23: 103–120.
- 14. Pennartz C. M, Groenewegen H. J, Lopes da Silva F. H (1994) The nucleus accumbens as a complex of functionally distinct neuronal ensembles: an integration of behavioural, electrophysiological and anatomical data. Prog Neurobiol 42: 719–761.
- 15. Cardinal R. N, Parkinson J. A, Hall J, Everitt B. J (2002) Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev 26: 321–352.
- 16. Everitt B. J, Robbins T. W (2005) Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci 8: 1481–1489.
- 17. Kalivas P. W, Volkow N. D (2005) The neural basis of addiction: a pathology of motivation and choice. Am J Psychiatry 162: 1403–1413.
- 18. Sturm V, Lenartz D, Koulousakis A, Treuer H, Herholz K, et al. (2003) The nucleus accumbens: a target for deep brain stimulation in obsessive-compulsive- and anxiety-disorders. J Chem Neuroanat 26: 293–299.
- 19. Ito R, Robbins T. W, Pennartz C. M, Everitt B. J (2008) Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning. J Neurosci 28: 6950–6959.
- 20. Pavlides C, Winson J (1989) Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes. J Neurosci 9: 2907–2918.
- 21. Wilson M. A, McNaughton B. L (1994) Reactivation of hippocampal ensemble memories during sleep. Science 265: 676–679.
- 22. Hoffman K. L, McNaughton B. L (2002) Coordinated reactivation of distributed memory traces in primate neocortex. Science 297: 2070–2073.
- 23. Pennartz C. M, Lee E, Verheul J, Lipa P, Barnes C. A, et al. (2004) The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J Neurosci 24: 6446–6456.
- 24. Ji D, Wilson M. A (2007) Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat Neurosci 10: 100–107.
- 25. Marr D (1971) Simple memory: a theory for archicortex. Philos Trans R Soc Lond B Biol Sci 262: 23–81.
- 26. Buzsaki G (1989) Two-stage model of memory trace formation: a role for “noisy” brain states. Neuroscience 31: 551–570.
- 27. McClelland J. L, McNaughton B. L, O'Reilly R. C (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102: 419–457.
- 28. McNaughton B. L, Barnes C. A, Battaglia F. P, Bower M. R, Cowen S. L, et al. (2003) Off-line reprocessing of recent memory and its role in memory consolidation: a progress report. In: Maquet P, Smith C, Stickgold R, editors. Sleep and brain plasticity. New York: Oxford University Press. pp. 225–246.
- 29. Skaggs W. E, McNaughton B. L (1996) Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271: 1870–1873.
- 30. Lee A. K, Wilson M. A (2002) Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36: 1183–1194.
- 31. Lansink C. S, Goltstein P. M, Lankelma J. V, Joosten R. N, McNaughton B. L, et al. (2008) Preferential reactivation of motivationally relevant information in the ventral striatum. J Neurosci 28: 6372–6382.
- 32. Mehta M. R, Lee A. K, Wilson M. A (2002) Role of experience and oscillations in transforming a rate code into a temporal code. Nature 417: 741–746.
- 33. O'Keefe J, Recce M. L (1993) Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus 3: 317–330.
- 34. Skaggs W. E, McNaughton B. L, Wilson M. A, Barnes C. A (1996) Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6: 149–172.
- 35. Qin Y. L, McNaughton B. L, Skaggs W. E, Barnes C. A (1997) Memory reprocessing in corticocortical and hippocampocortical neuronal ensembles. Philos Trans R Soc Lond B Biol Sci 352: 1525–1533.
- 36. Kudrimoti H. S, Barnes C. A, McNaughton B. L (1999) Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. J Neurosci 19: 4090–4101.
- 37. Buzsaki G (1986) Hippocampal sharp waves: their origin and significance. Brain Res 398: 242–252.
- 38. Jones M. W, Wilson M. A (2005) Theta rhythms coordinate hippocampal-prefrontal interactions in a spatial memory task. PLoS Biol 3: e402.
- 39. DeCoteau W. E, Thorn C, Gibson D. J, Courtemanche R, Mitra P, et al. (2007) Learning-related coordination of striatal and hippocampal theta rhythms during acquisition of a procedural maze task. Proc Natl Acad Sci U S A 104: 5644–5649.
- 40. Tran A. H, Tamura R, Uwano T, Kobayashi T, Katsuki M, et al. (2005) Dopamine D1 receptors involved in locomotor activity and accumbens neural responses to prediction of reward associated with place. Proc Natl Acad Sci U S A 102: 2117–2122.
- 41. Euston D. R, Tatsuno M, McNaughton B. L (2007) Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science 318: 1147–1150.
- 42. Aertsen A. M, Gerstein G. L, Habib M. K, Palm G (1989) Dynamics of neuronal firing correlation: modulation of “effective connectivity.” J Neurophysiol 61: 900–917.
- 43. Eggermont J. J (1992) Neural interaction in cat primary auditory cortex. Dependence on recording depth, electrode separation, and age. J Neurophysiol 68: 1216–1228.
- 44. Carr G. D, White N. M (1983) Conditioned place preference from intra-accumbens but not intra-caudate amphetamine injections. Life Sci 33: 2551–2557.
- 45. Kelley A. E, Domesick V. B (1982) The distribution of the projection from the hippocampal formation to the nucleus accumbens in the rat: an anterograde- and retrograde-horseradish peroxidase study. Neuroscience 7: 2321–2335.
- 46. Markram H, Lubke J, Frotscher M, Sakmann B (1997) Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275: 213–215.
- 47. Abbott L. F, Nelson S. B (2000) Synaptic plasticity: taming the beast. Nat Neurosci 3: Suppl1178–1183.
- 48. Schultz W, Dayan P, Montague P. R (1997) A neural substrate of prediction and reward. Science 275: 1593–1599.
- 49. Sutton R. S, Barto A. G (1998) Reinforcement learning: an introduction. Cambridge (Massachusetts): MIT Press. 322 p.
- 50. Rescorla R. A, Wagner A. R (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black A. H, Prokasy W. F, editors. Classical conditioning II: current research and theory. New York: Appleton Century Crofts. pp. 64–99.
- 51. Lansink C. S, Bakker M, Buster W, Lankelma J, van der Blom R, et al. (2007) A split microdrive for simultaneous multi-electrode recordings from two brain areas in awake small animals. J Neurosci Methods 162: 129–138.
- 52. Paxinos G, Watson C (1996) The rat brain in stereotaxic coordinates. 3rd edition. New York: Academic Press. 237 p.
- 53. Tatsuno M, Lipa P, McNaughton B. L (2006) Methodological considerations on the use of template matching to study long-lasting memory trace replay. J Neurosci 26: 10727–10742.
- 54. Kleinbaum D. G, Kupper L. L, Muller K. E, Nizam A (1998) Applied regression analysis and other multivariate methods. 3rd edition. London: Duxbury Press. 798 p.
- 55. Sokal R. R, Rohlf F. J (1995) Biometry: the principles and practice of statistics in biological research. 3rd edition. New York: WH Freedman. pp. 823–825.
- 56. Skaggs W. E, McNaughton B. L, Gothard K. M (1992) An information-theoretic approach to deciphering the hippocampal code. In: Hanson S, Cowan J, Giles L, editors. Neural information processing systems. San Francisco: Morgan Kaufmann. pp. 1030–1037.
- 57. Panzeri S, Treves A (1996) Analytical estimates of limited sampling biases in different information measures. Network 7: 87–107.
- 58. Perkel D. H, Gerstein G. L, Moore G. P (1967) Neuronal spike trains and stochastic point processes. II. Simultaneous spike trains. Biophys J 7: 419–440.