The Time Course of the Influence of Valence and Arousal on the Implicit Processing of Affective Pictures

In the current study, we investigated the time course of the implicit processing of affective pictures with an orthogonal design of valence (negative vs. positive) by arousal (low vs. high). Previous studies with explicit tasks suggested that valence mainly modulates early event-related potential (ERP) components, whereas arousal mainly modulates late components. However, in this study with an implicit task, we observed significant interactions between valence and arousal at both early and late stages over both parietal and frontal sites, which were reflected by three different ERP components: P2a (100–200 ms), N2 (200–300 ms), and P3 (300–400 ms). Furthermore, there was also a significant main effect of arousal on P2b (200–300 ms) over parieto-occipital sites. Our results suggest that valence and arousal effects on implicit affective processing are more complicated than previous ERP studies with explicit tasks have revealed.


Introduction
As one of the fundamental aspects of the human mind, emotion and its brain mechanisms have received increasing interest recently, especially with the development of non-invasive neuroimaging techniques such as functional magnetic resonance imaging (fMRI) and event-related potentials (ERPs) [1]. Compared to fMRI, the ERP technique can provide fine-grain temporal resolution of mental processing and thus has been widely used in studying the time course of affective processing. Studies have revealed that affective modulations can occur as early as around 100 ms after stimulus onset and can be sustained for as long as several seconds [2]. According to the dimensional view of emotion, affective valence (ranging from negative to positive) and arousal (ranging from calm to excited) are two primary dimensions of emotion [3]. A rich body of literature has suggested that these two dimensions modulate ERP components at different temporal stages with valence mainly affecting ERP components at an early stage (e.g., before 200 ms) and arousal mainly affecting ERP components at a later stage (e.g., after 250 ms) [2,4,5].
The modulating effect of valence on the early stage of affective processing has been attributed to the biological significance of negative stimuli [2,6]. At the early stage, several ERP components (e. g., P1, N1, P2) elicited by negative stimuli have been found to be more pronounced than those elicited by positive and neutral stimuli [7,8,9,10]. However, such valence effects are not consistent among previous studies. For instance, comparable early ERP components elicited by negative and positive stimuli have also been reported [5,11,12,13,14]. Additionally, a considerable number of studies have even reported that early ERP components elicited by positive stimuli can be more pronounced than those elicited by negative stimuli, possibly due to the greater ''automatic arousal'' of positive stimuli [12,15,16,17].
In contrast, an association between arousal effects and late ERP components is prevalent in previous studies; in particular, arousal effects usually occurred at around 250 ms and were accentuated over the following several seconds [15,18,19]. For instance, high arousing (negative and positive) stimuli have been found to elicit more pronounced late ERP components (e. g., N2, P3) than low arousing (neutral) stimuli, even when the affective stimuli were briefly presented [5,12,14,15,20,21]. The modulating effects of arousal on late ERP components were suggested to reflect motivation-driven attention to high arousing stimuli, indicating that meaningful stimuli are selectively processed at the late stage [22]. However, such arousal effects have not been consistently observed, and valence effects on late ERP components have also been reported [23,24,25]. Current evidence is thus unclear as to whether valence, arousal, or both factors together modulate the late processing of affective stimuli [18].
Although valence and arousal have been suggested to modulate ERP components at relatively separate stages [2], recent behavioral and fMRI studies have found that the effects of arousal and valence are more complicated than the ERP studies have suggested, especially in their interactions [17,26,27,28,29,30,31]. For example, in a behavioral study, Robinson et al. found that participants responded faster to high arousing negative and low arousing positive stimuli than to low arousing negative and high arousing positive stimuli. Subsequent studies further demonstrated that the interaction between valence and arousal can be elicited automatically [32]. Such an interaction has been further confirmed in recent fMRI studies. In a study assessing the neural responses to affective items which are successfully recognized, Steinmetz and his colleagues found that high arousing negative and low arousing positive stimuli enhanced connections between the amygdala and other regions, whereas low arousing negative and high arousing positive stimuli reduced these connections [30]. One way to interpret the valence by arousal interaction is using the model of evaluative space proposed by Cacioppo and his colleagues [33,34,35,36,37,38]. According to this model, motivational systems for positive and negative valence processing (appetitive system and aversive system, respectively) are separable and are characterized by different activation functions [39]: whereas the appetitive motivational system tends to respond more than the aversive motivational system when the evaluative input (as indicated by arousal ratings) is low (positive offset), the aversive motivational system responds more than appetitive motivational system when the evaluative input is high [36,38]. This occurs because the aversive motivational system tends to response more intensely than appetitive motivational to comparable increases in input (negative bias). The positive offset and negative bias could well explain the valence by arousal interaction found in behavioral and fMRI studies [29,30,32,40], as well as the inconsistent valence effects found in previous ERP studies.
Whereas interactions between valence and arousal have been amply demonstrated in behavioral and fMRI studies, there is little evidence for such interactions from ERP studies [2,17]. A number of ERP studies have been conducted with orthogonal factors of valence by arousal [4,5,17,18,20,41], but only one of these revealed a significant interaction between valence and arousal, which was found by analyzing ERP topographies rather than ERP components [4].
Previous ERP studies investigating affective processing often used explicit tasks such as the passive viewing task, oddball paradigm, and affective categorization task [4,17,18,19,23,24]. However, numerous studies have shown that affective content can be processed even when it is not explicitly attended [42,43,44,45,46]. For example, when participants are instructed to categorize affective pictures according to nonaffective features of these stimuli (e.g., identifying how many people in the pictures), affective information has been found to modulate ERP components at both early and late stages [7,8,10,45,47,48]. Nevertheless, how such implicit affective processing can be influenced by valence and arousal and what are the corresponding ERP correlates are yet under debate [8,43,48]. For instance, previous studies have frequently associated P300 with arousal in implicit affective processing [47,48,49], but Carretie and his colleagues [43,44] demonstrated that the valence-associated N300 (275-325 ms) might be a better index than P300 to reflect implicit processing of affective stimuli.
In sum, the exact contributions of valence and arousal in affective processing are unclear in previous studies, and an orthogonal design of valence by arousal would provide a promising means to systematically investigate their independent and interactive effects on affective processing.
In the present study, we aimed to investigate the electrophysiological correlates of the implicit processing of affective stimuli. Compared to previous implicit tasks in which participants categorized affective stimuli according to nonaffective features of pictures [7,8,43,46,47,49,50,51], our task was even more ''implicit'' in that participants did not even need to attend to and process the content of the pictures but merely needed to judge the color of the picture frame [10,52]. In addition, we employed an orthogonal design of valence by arousal so that both the main effects and interaction could be systematically investigated. According to Cacioppo et al.'s model and prior behavioral findings [29,32], both faster RTs and higher amplitude ERPs might reflect enhanced responses to the appetitive or aversive motivational system [53,54]. Accordingly, we hypothesized that when the arousal ratings of stimuli were low, positive stimuli would elicit larger ERPs than negative stimuli, whereas when arousal ratings were high, positive stimuli would elicit smaller ERPs than negative stimuli.

Participants
Twenty-one individuals (10 females) aged 18-29 years (mean age = 21 years) participated in the present study for payment. All participants had normal or corrected-to-normal vision and did not have any history of psychiatric or neurological illness. This study and the recruitment of participants were approved by the Beijing Normal University Institutional Review Board (IRB). Written informed consents were collected for all participants.

Stimuli
Pictures used in our study were selected from the International Affective Picture System based on normative ratings of valence and arousal [3]. There were four stimuli types: high arousing positive (HP), low arousing positive (LP), high arousing negative (HN), and low arousing negative (LN), with 15 Table 1. Positive and negative pictures were significantly different in valence ratings (p,.001), but not in arousal ratings (p = .41); whereas high arousing and low arousing pictures were significantly different in arousal ratings (p,.001), but not in valence ratings (p = .55). All pictures were adjusted to a size of 12u69u and were surrounded on a random basis by one of four colored frames (red, yellow, blue, or green) having a width of 0.5u. Pictures were presented on a CRT monitor with an 80 Hz refresh rate using the Psychophysics Toolbox extensions for Matlab [55,56].

Procedure
The experimental procedure is shown in Figure 1, each trial began with a fixation (750 ms) point, followed by the stimulus for 300 ms. The inter-trial interval was randomized between 1700 and 2300 ms. Participants were instructed to judge the color of the frame as quickly and accurately as possible by pressing one of four corresponding keys. The whole experiment consisted of four blocks of each of the four stimulus types [13,57,58,59]. Each block contained 75 trials. The orders of four blocks were randomized across participants, and the sequence of trials in each block was pseudo-random, with the constraints that identical pictures were not presented successively and picture frames with the same color were not presented successively more than three times. Participants sat in a dimly lit and sound-attenuated chamber facing a CRT monitor 80 cm away from their eyes. Before the formal experiment, each participant was given a training block to get familiar with response keys. Four pictures that were not included in the formal experiment were used in a 20 trial of training block.

Electrophysiological recording
The electroencephalogram (EEG) was recorded from 64 scalp sites using electrodes mounted in an elastic cap (Brain Product, GmbH, Germany), with an online reference to the left mastoid. The horizontal electroencephalogram (HEOG) was recorded with two electrodes placing laterally to the right and left eyes. The vertical electroencephalogram (VEOG) was recorded with electrodes placed above and below the right eye. All inter-electrode impedances were maintained below 10 kV. The EEG and EOG were amplified using a 0.01-100 Hz band-pass and continuously sampled at 500 Hz in each channel for off-line analysis. All EEG signals were re-referenced off-line to the average of left and right mastoids [60]. The EEG data were low-pass filtered below 30 Hz (24 dB/oct) and were corrected for eye movements or blinks using the Gratton and Coles [61] method as implemented in the Brain Vision analysis software (Brain Product, GmbH, Germany). Trials containing EEG sweeps with amplitudes exceeding 6100 mV were excluded.

Data reduction and analysis
ERPs elicited by each type of stimulus were averaged separately over an epoch of 800 ms with a 200 ms pre-stimulus baseline. Based on inspection of the grand-averaged ERP waveforms ( Figure 2) and previous studies of affective picture processing [7,13,14,22], four different ERP components, P2a, P2b, N2, and P3 were analyzed in the current study. Different sets of electrodes in the frontal and parietal regions were chosen for the baseline-topeak measurements of these components: FC3, FCz, FC4, F3, Fz, and F4 for P2a (100-200 ms) and N2 (200-300 ms) components; PO3 and PO4 for the P2b (200-300 ms) component; P1, P3, P2 and P4 for the P3 (300-400 ms) component. The amplitudes and latencies of these components were then analyzed using repeated measures ANOVAs with the factors of Valence (negative vs. positive) by Arousal (low vs. high) by Hemisphere (left vs. right) by Electrodes (electrodes selected for particular ERP component, as stated above). P values were adjusted according to the Greenhouse-Geisser correction if necessary. Bonferroni correction was used for multiple comparisons.

Behavioral results
The behavioral results are shown in Table 1 49), such that P2a amplitudes elicited by high arousing negative pictures were larger than those by high arousing positive pictures; whereas the P2a amplitudes elicited by low arousing negative pictures were smaller than those by lowing arousing positive pictures (Figures 2 and 3, Table 2). Such a Valence by Arousal interaction has rarely been reported in previous ERP studies in which the valence effect was related to this early stage [2,51]. Further analysis also revealed that the P2a amplitudes were larger at F3, Fz, and F4 (M = 7.82 mV, SD = 4.19) than at FC3, FCz, and FC4 (M = 6.33 mV, SD = 3.69). Four-way repeated measures ANOVA of Valence by Arousal by Hemisphere by Electrode on P2a latencies (Table 3)   hemisphere, N2 amplitudes were more negative for low arousing negative pictures than for high arousing negative pictures; whereas N2 amplitudes elicited by high and low arousing positive pictures showed no significant difference. In addition, N2 amplitudes were more negative for low arousing negative pictures than for low arousing positive pictures; whereas N2 amplitudes elicited by high arousing negative and positive pictures showed no significant difference (Figure 2 and 3, Table 2). In contrast, over the left and medial sites, no effects were found on the N2 amplitudes. These results indicated that the arousal effects on N2 depended on valence, and such an interaction was restricted to the right hemisphere.  Table 2). Neither the main effect of Valence (F(1, 20) = .13, p = .72) nor the interaction between Valence and Arousal (F(1, 20) = 3.74, p = .067) was significant. These results indicated that the arousal information of affective stimuli could be independently processed within this stage [14,22]. Three-way repeated measures ANOVA of Valence by Arousal by Hemisphere on P2b latencies (Table 3) revealed a marginally significant main effect of Arousal (F(1, 20) = 4.50, p = .047,g 2 p = .18), and a three-way interaction of Valence, Arousal, and Hemisphere (F(1, 20) = 4.45, p = .048, g 2 p = .18), such that the latencies for high arousing negative pictures were shorter than those for low arousing negative pictures over the left hemisphere. These results were consistent with previous studies showing that P2b latencies were shorter for low arousing stimuli than for high arousing stimuli [63].  hemisphere. Interestingly, P3 showed a Valence by Arousal interaction similar to P2a over the right hemisphere, such that the P3 amplitudes for high arousing negative pictures were larger than for high arousing positive pictures; whereas P3 amplitudes elicited by low arousing negative pictures were smaller than those by low arousing positive pictures (Figure 2 and 4, Table 2). Such an interaction was not found over the left hemisphere. Such a Valence by Arousal interaction pattern on ERP components has rarely been previously reported, although the interaction of valence and arousal on affective processing at this stage has been suggested [4,41]. Four-way repeated measures ANOVA of Valence by Arousal by Hemisphere by Electrode on P3 latencies (Table 3)  and P2 electrodes were shorter than those at P3 and P4 electrodes. Such results of P3 latencies are consistent with previous studies showing that latency reductions at posterior sites were restricted to negative stimuli [64].

Discussion
In the present study, we investigated the influence of valence and arousal on the implicit processing of affective pictures using the ERP technique. We found significant interactions between valence and arousal within both early and late time windows over both parietal and frontal sites, which were reflected in three different ERP components: P2a (100-200 ms), N2 (200-300 ms), and P3 (300-400 ms). In addition, we also found a significant main effect of arousal on the P2b (200-300 ms) component over parieto-occipital sites. These results indicate that valence and Specifically, valence-arousal-congruent pictures (HN and LP) elicited larger P2a than valence-arousal-incongruent pictures (LN and HP). In addition, the arousal effects for N2were restricted to negative pictures, showing that LN elicited larger N2 than HN. Moreover, the interaction between Valence and Arousal on N2 was restricted to the right hemisphere. doi:10.1371/journal.pone.0029668.g003  arousal play interactive roles in the implicit processing of affective stimuli. An interaction between valence and arousal effects was found on both the early P2a and late P3 components, such that high arousing negative (HN) and low arousing positive (LP) pictures elicited larger P2a and P3 amplitudes than high arousing positive (HP) and low arousing negative (LN) pictures. These results are consistent with previous behavioral results showing that participants responded faster to HN and LP stimuli than to HP and LN stimuli [29]. Robinson and his colleagues [29] proposed that because high arousing stimuli, no matter whether positive or negative, are negatively coded, the HN and LP stimuli are thus congruent, whereas HP and LN stimuli are incongruent in the valence and arousal dimensions. The observed RT differences could be explained by assuming that valence-arousal-incongruent stimuli (HP and LN) are evaluated more slowly than valencearousal-congruent stimuli (HN and LP). In another study with an affective Simon task, Eder and Rothermund further demonstrated that the congruency of valence and arousal dimensions can affect processings of stimuli even when participants do not attend to their affective contents. They thus concluded that arousal interacts with valence in stimulus processing, and such interactions occur preattentively [32]. In agreement with these behavioral studies, a recent ERP study found that the interaction between valence and arousal can occur within the interval 302-330 ms [4]. Using the ERP microstate analysis of topography, Gianocci and colleagues found an interesting ''common step'' between the processing of valence and arousal, such that the last microstate of valence extraction was identical to the first microstate of arousal extraction. This microstate lasted from 302 to 330 ms after the stimulus onset. The authors suggested that such a ''common step'' reflects the interaction between valence and arousal on affective processing [4].
Our results support and extend previous behavioral findings showing reaction time differences between HN & LP and HP & LN stimuli [29]. Our ERP results indicate that these behavioral differences might be associated with the P2a and P3 components. These results suggest that HN & LP stimuli evoke more affective processing at both early and late stages than HP & LN stimuli.
One possible theoretical explanation for our finding is based on the model of evaluative space proposed by Cacioppo et al. [33,34]. According to this model, stimuli of low arousal will generate a positive offset that suggests a larger response to positive stimuli (positive offset), whereas stimuli of high arousal will generate an enhanced response to negative stimuli (negativity bias). Both positive offset and negativity bias have been observed in previous behavioral studies [29,32] (but see [65] for exception), but only negativity bias has been emphasized in ERP studies [9,66]; such valence effects in ERPs have been considered inconsistent, however [2]. Our results supported and extended previous behavioral findings by showing that positive offset and negativity bias effects can occur at both early (P2a) and later (P3) stages of processing.
In addition, we found a significant interaction between valence and arousal on the N2 component, but with a different pattern than P2a and P3, such that N2 amplitudes were larger for LN than LP, but were identical for HN and HP (Fig.3). Moreover, such an interaction was restricted to the right hemisphere. In previous The main effect of Arousal was only found on P2b, such that high arousing pictures elicited larger P2b than low arousing pictures. Interaction between Valence and Arousal was found on P3. Specifically, valencearousal-congruent pictures (HN and LP) elicited larger P3 than valence-arousal-incongruent pictures (LN and HP). In addition, the interaction between Valence and Arousal on P3 was restricted to the right hemisphere. doi:10.1371/journal.pone.0029668.g004 studies, N2 has been considered as a complex component comprised of several sub-components, some reflecting automatic affective processing and others reflecting executive control processing [10]. The N2 effect found in the current study might be related to both of these. On the one hand, previous studies showed that the arousal effect can automatically modulate ERP components within 200-300 ms, even when the cognitive resource is limited [2,14,67]. For instance, compared to high arousing negative and positive pictures, low arousing neutral pictures elicit a negative going component within 200-300 ms at frontal sites [68,69], even when pictures are briefly presented [14,22]. Olofsson et al. proposed that the arousal effect between 200-300 ms is related to the amygdala's response to affective stimuli. In our study, we found that low arousing pictures elicited a more negative N2 than high arousing pictures; such arousal effects, however, were only found for negative stimuli [69]. These results suggest that the interaction between valence and arousal on the amygdala might occur within 200-300 ms. On the other hand, the N2 effects found in our study might reflect the valence-related influence on affective processing that has been attributed to the exertion of cognitive control on threat-related stimuli [10,70,71]. Consistent with this view, a recent fMRI study found that negative pictures elicit stronger dorsolateral prefrontal activation than positive pictures when participants perform an implicit task [70]. Additionally, the N2 effects we found were prevalent over the right hemisphere, which accords with previous studies showing that negative affective processing is primarily a function of the right hemisphere [67,72,73,74]. In either case, our study demonstrated that valence and arousal also played an interactive role in implicit affective processing at this stage.
In addition to the above-described interactions between valence and arousal, we also found a significant main effect of arousal on the posterior P2b component, such that high arousing pictures elicited larger P2b than low arousing pictures. Such an arousal effect has also been reported in previous studies [50,75].
In reviewing ERP studies of affective processing in recent decades, Olofsson et al. suggested that valence modulates early ERP components and arousal modulates late ERP components [69]. The present results with implicit affective processing suggest that their proposal might be oversimplified. Two possible reasons might account for why our results were different from previous findings in emotion evaluation tasks. Firstly, participants in previous emotion evaluation tasks were asked to categorize emotional pictures, or simply to look at the emotional pictures. Such manipulations are likely to lead participants to assume that several kinds of affective stimuli (e.g., negative, high arousing stimuli) are more relevant to the task than others [7,43,44,75]. Such a ''relevance-for-task effect'' has been found to modulate ERP components [44]. Thus, implicit tasks (such as the one used in the present study) might provide a better way to study the valence and arousal effects related to affective processing, because they could avoid the interference of the ''relevance-for-task effect''. Secondly, according to the model of evaluation space proposed by Cacioppo [33,76], when the evaluative activation is low, the intrinsic response of an individual to affective stimuli is that the motivation to explore is stronger than the motivation to avoid (positive offset). However, in previous emotion evaluation tasks (especially in the categorization task), participants were instructed to categorize affective pictures according to their valence, which means they were probably motivated to explore every presented picture. Such a manipulation might cause ''task-related motivation'', which would influence the individual's intrinsic response to affective pictures. In contrast, participants in implicit tasks are asked to do tasks that are unrelated to the affective picture. Such a manipulation would avoid interference from ''task-related motivation''; therefore the individual's intrinsic response to affective stimuli would be more likely to be observed in implicit tasks. In summary, using implicit tasks could avoid the ''relevance-for-task effect'' and ''task-related motivation'' that might interfere with an individual's intrinsic response to affective stimuli; this might explain why the results observed in the present study were different from prior findings with emotion evaluation tasks.
A limitation of the present study is that we did not screen participants in terms of possible anxiety or depressive states. Although previous behavioral and fMRI studies have observed that anxiety and depressive states modulate the processing of affective pictures [77,78,79,80,81], such influence has been frequently ignored in ERP studies [2,17,48,51]. Several recent studies have indicated that ERPs elicited by affective pictures can be modulated by anxiety states [82,83,84], suggesting a promising approach to characterize clinical affective dysfunction [2]. Thus, the modulations of anxiety and depressive states on ERPs evoked by affective pictures deserve systematic investigation in future studies. Another limitation of the current study is that we did not add a blank screen between fixation and picture (see Figure 1) to avoid possible contamination of the ERPs evoked by pictures by a fixation offset effect. Although such an offset effect is generally weak [85], and similar stimulus sequences have been used in previous studies [82,84], this experimental design flaw should be avoided in future studies.