The relationship between contextual information and object perception has received considerable attention in neuroimaging studies. In the work reported here, we used functional magnetic resonance imaging (fMRI) to investigate the relationship between aesthetic judgment and images of objects in their normal contextual setting versus images of objects in abnormal contextual settings and the underlying brain activity. When object-context relationships are violated changes in visual perception and aesthetic judgment emerges that exposes the contribution of vision to interpretations shaped by previous experience. We found that effects of context on aesthetic judgment modulates different memory sub-systems, while aesthetic judgment regardless of context recruit medial and lateral aspects of the orbitofrontal cortex, consistent with previous findings. Visual cortical areas traditionally associated with the processing of visual features are recruited in normal contexts, irrespective of aesthetic ratings, while prefrontal areas are significantly more engaged when objects are viewed in unaccustomed settings.
Citation: Kirk U (2008) The Neural Basis of Object-Context Relationships on Aesthetic Judgment. PLoS ONE 3(11): e3754. https://doi.org/10.1371/journal.pone.0003754
Editor: Bernhard Baune, James Cook University, Australia
Received: March 31, 2008; Accepted: October 29, 2008; Published: November 19, 2008
Copyright: © 2008 Kirk. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: U. Kirk was supported by a PhD scholarship from the Danish Medical Research Council. S. Zeki was funded by a grant from the Wellcome Trust. No financial interests influenced this experiment.
Competing interests: The author has declared that no competing interests exist.
Objects are usually associated with the context in which they appear, which provide expectations about which objects are likely to appear in a given contextual scene. The importance of prior knowledge in determining how contextual association influences object recognition has been well-documented in behavioural studies. These studies show that objects appearing in familiar or consistent contextual scenes are detected more accurately and processed more quickly than objects appearing in unfamiliar or inconsistent contexts –. This suggests that objects and the context in which they appear are processed interactively, thus facilitating the perceptual processes involved in visual object recognition , . Such behavioural findings have recently been adapted to neuroimaging studies –. Bar and colleagues (2003) investigated the cortical correlates of contextual associations by comparing objects with strong contextual associations presented in their normal surroundings (e.g. a blender) with objects with weak contextual associations (e.g. a mobile-phone). They found that the former elicited greater activity than the latter in the parahippocampal gyrus and in the retrosplenial cortex, which together comprise a cortical network processing contextual associations during object recognition , .
In extending this work we wanted to investigate the neural effects of context (normal and abnormal) on aesthetic judgment. Such an approach promised us insights in the general field of neuroaesthetics. Indeed the stimuli that we prepared for this study were inspired by the work of the Surrealist artist René Magritte (1898–1967) who, in his compositions, gave visual objects primacy by divorcing them from the surroundings with which they are usually associated. Likewise, stimulus novelty has been shown to have a significant effect on aesthetic ratings , . Because the effect of perceptual novelty is so explicit in the work of Magritte, the use of stimuli derived from this approach allowed us to analyse the influence of novelty on aesthetic ratings.
In the present study, the critical components of interest were objects that were either consistent or inconsistent with their normally associated context (fig. 1), in an event-related fMRI approach. As object-context relationships are learnt over time, based on real life pairing frequencies , conflicting or abnormal trials can be experimentally regulated by altering the object-context pairings into less probable ones. Thus, by varying the setting in which objects appeared, we were able to assess the neural response to the relationships between the two relative to the aesthetic ratings attributed to each stimulus. Previous work has shown that perceived beauty correlates with activity in the medial aspects of the orbito-frontal cortex (OFC), with the intensity of activity there being related to the declared experience of the beauty of the viewed work of art . We wanted to supplement this work by examining the relationship between the declared experience of the aesthetic satisfaction derived from viewing the images we presented, whether in normal or abnormal contexts, and activity in the relevant cortical areas. To accommodate this experimental aim we applied linear parametric contrasts and interactions to investigate effects of normal and abnormal context on aesthetic ratings.
Examples of the stimulus material used in the fMRI experiment are shown; the stimulus material was constructed in pairs such that objects were presented in both a normal and an abnormal contextual setting. Stimulus presentation was 3500 ms followed by a response-period (black screen) where subjects were required to indicate their aesthetic rating (appealing, neutral, unappealing) via button press (2000 ms).
Additionally, the experimental design allowed us to test whether the neural pathways engaged when we view objects in normal settings are significantly different from those engaged when we view them in abnormal ones, irrespective of aesthetic ratings. The chances that this would be so seemed high, given that departures from expectation in general results in activation of areas that are not found to be active when the percept is consistent with prior experience. In particular, many studies have shown that the dorso-lateral prefrontal cortex (DLPFC) and especially Brodmann area 46 is active when subjects experience stimuli, whether visual or otherwise, that depart from expectation. Among such stimuli are objects dressed in un-natural colours , oddball tasks , infrequent events –, general perceptual and emotional deviance  and irregular temporal patterns . Our general hypothesis was, therefore, that when objects are viewed in unusual or unnatural settings, the activity will always involve the DLPFC. Furthermore, as subjects were instructed to rate each stimulus according to aesthetics it encourages subjects, although implicitly, to ignore contextual information. This experimental design extends previous studies that used behavioural tasks where subjects were required to attribute explicit attention to contextual information in the stimuli-material , . Specifically, we were able to investigate whether the cortical context network , such as the parahippocampal gyrus and the retrosplenial cortex, would be influenced by this change.
Materials and Methods
Fifteen right-handed subjects (six females) with a mean age of 24.4 years participated in the study. All gave informed consent in accordance with the Declaration of Helsinki, and the Ethics Committee of the National Hospital for Neurology and Neurosurgery, London, UK, granted ethics approval for the study. All had normal or corrected-to-normal vision, and none had a history of neurological or psychiatric disorders. Subjects were all undergraduate or graduate students.
Stimuli and task
Visual chromatic stimuli belonging to two categories, normal and abnormal photographs, 120 in total, were selected from online sources. Abnormal versions were manipulated in Photoshop (version 7.0, Adobe, USA) by superimposing a photo of an object onto a different contextual background such that the object was in the foreground and appeared against a background with which the object is not usually associated (abnormal conditions) (fig. 1). Any image noticeably distorted (e.g. in proportion or illumination) by this process was excluded from the stimulus pool. The normal versions consisted of photos with an object against a background with which it is normally associated (normal conditions). The stimulus material was paired, such that the same objects were presented in normal and abnormal context. The experimental protocol consisted of an event-related design in which subjects were scanned while being presented with each of the stimuli only once. All trials were presented in a pseudorandom order, and trials were counterbalanced across subjects meaning that half of the subjects saw the abnormal version of a given stimuli before its normal counterpart, while this was reversed for the other half of subjects. A trial began with a fixation cross for 500 ms positioned at the centre of the screen against a black background. Each stimulus was presented at the centre of a computer screen on a black background for 3500 ms. This was followed by a response-period of 2000 ms (black screen) in which subjects were instructed to decide whether they found each stimulus aesthetically ‘appealing’, ‘unappealing’ or ‘neutral’. Subjects were instructed to passively view each image during the 3500 ms stimulus presentation. They could make their rating at any time during the response-period, by pressing one of three buttons on a response key-pad corresponding to the rating options with their right hand. Prior to scanning, subjects were instructed in the aesthetic rating task and were subsequently trained using demonstration stimuli, which were not included in the scanning session, until they were accustomed with the aesthetic rating task. It was imperative that the aesthetic rating task was presented in real-time (i.e. during scanning) in order to measure the oddball-effect on aesthetic ratings. The alternative would be to present the aesthetic rating task either pre- or post-scanning. However, this would inevitably have introduced a bias by compromising the oddball-effect (i.e. abnormal conditions). The stimuli were presented at a screen resolution of 1024×768 pixels displayed at a visual angle of 24×18°, and centred in a 500×500 pixel resolution surrounded by a black background. Stimuli were presented and responses collected using COGENT 2000 Graphics (http://www.vislab.ucl.ac.uk/Cogent/) running in Matlab (Mathworks Inc.). Total scanning-time per subject was 12 min. in one session. The stimuli were back-projected via a LCD projector onto a transparent screen positioned over the subjects' head and viewed through a tilted mirror fixed to the head coil.
The functional imaging was conducted by using a 1.5T Siemens Vision fMRI scanner (Siemens, Erlangen, Germany) to acquire gradient T2* weighted echo planar images (EPI) to maximize the blood oxygen level-dependent (BOLD) contrast (TE, 40 ms; TR, 3.42 s; flip angle, FA = 90°). Each functional image was acquired in a descending sequence comprising 38 axial slices each 2.5 mm thick, consisting of 64×64 voxels. This gave near whole-brain coverage, excluding the cerebellum. Each session consisted of 220 volumes with the first 5 volumes being discarded to allow for T1 equilibration effects. After every functional scan, a T1 weighted structural sequence was acquired, using a phased array head coil to provide high-resolution anatomical detail. The structural image was co-registered to the EPI images, thus allowing functional data to be overlaid on a high-resolution anatomical image.
Image pre-processing and data analysis was performed using SPM2 (Wellcome Department of Imaging Neuroscience, London, UK). The EPI images were realigned spatially . This was followed by temporal realignment, which corrected for slice-time differences using the middle slice as reference. Images were then normalized to the Montreal Neurological Institute (MNI) template provided in SPM2. Finally a spatial filtering was performed by applying a Gaussian smoothing kernel of 12 mm full width at half-maximum.
Following pre-processing a general linear model was applied to the fMRI time-series where stimulus onset was modeled as single impulse response function and convolved with SPM2's canonical haemodynamic response function. Data from 15 subjects were applied in a random-effects (RFX) multiple regression analysis, whereby the reliability of the measurements was assessed in relation to the between subject variance . We modelled six regressors of interest: abnormal-appealing, abnormal-neutral, abnormal-unappealing, normal-appealing, normal-neutral and normal-unappealing. Button–press and question-duration times were modelled as regressors of no interest. Residual effects of head motion were corrected for by including the six estimated motion parameters for each subject as regressors of no interest. For the analysis a high pass filter with a cut-off frequency at 1/128 Hz was applied. As the behavioural data demonstrated a statistically significant interaction between stimulus category and rating category (fig. 2B), further regressors were applied to separate the data in order to control for these behavioural differences between conditions. For each subject the frequency of each of the three rating bins was balanced, such that there were no differences in ratings and reaction-times between normal and abnormal conditions as such a difference would confound the results with context-effects or aesthetic ratings or a combination of the two. The residual rating bins from this process were for each subject put in separate regressors.
(A) Mean aesthetic ratings for the two stimulus conditions, normal and abnormal, across subjects. (B) Distribution of aesthetic responses for the two stimulus conditions, normal and normal, across the three rating bins (appealing, neutral, and unappealing). (C) Mean reaction times (RT) for each of the two stimulus conditions. (D) Distribution of RTs for the two stimulus conditions across the three rating bins. (E) Mean aesthetic ratings for normal and abnormal conditions for each subject, compiled across stimuli. (F) Stimuli are shown distributed across the range of ratings. Stimuli are rank-ordered by mean aesthetic ratings separate for each condition, compiled across subjects. Error bars are standard deviation is all displays (A–F).
The resultant parameter estimates for each regressor (at each voxel) were compared using t-tests, allowing us to test for correlations of the fMRI BOLD signal and the parameters of interest. The statistical results given are based on a single-voxel t-statistics corresponding to p<0.001, uncorrected for multiple comparisons. In order to correct for multiple comparisons in interpreting these results, small volume corrections (SVC)  with a sphere of 10 mm radius were used for medial OFC of which we had prior anatomical hypothesis. The SVC was performed using the co-ordinates provided by a previous study for medial OFC . Before using SVC, we transformed coordinates given by this previous study from Talairach space to MNI space (www.mrc-cbu.cam.ac.uk). The co-ordinates of all activations are reported in MNI space.
Examination of the behavioural data collected during scanning showed that abnormal conditions were rated as being on average 2.05 (std = 0.1) and normal conditions were on average 2.15 (std = 0.08) on the aesthetic rating scale (1 = unappealing; 2 = neutral; 3 = appealing). Statistical analysis showed no significant difference between the two conditions (paired t = 0.88; df = 14; p>0.39) (fig. 2A). Subject-averaged reaction times (RT) were 764 ms (std = 13.6) for abnormal conditions and 818 ms (std = 44.4) for normal ones. The two conditions did not produce significant differences in RT (paired t = 1.37; df = 14; p>0.3) (fig. 2C). Thus, an analysis of the average ratings and RT results shows that subjects did not respond significantly different to abnormal and normal conditions.
We next inspected whether the frequency of each rating category differed across stimulus category (fig. 2B). A stimulus category (abnormal, normal)×rating category (appealing, neutral, unappealing) ANOVA revealed no significant main effect for stimulus category (ANOVA, F(1,1) = 0, p>0.99) or rating category (ANOVA, F(1,2) = 2.59, p>0.08). There was a significant interaction between stimulus category and rating category (ANOVA, F(1,2) = 9.15, p<0.0003).
Although subjects rated the normal and abnormal images as being on average equally appealing, they were more likely to attribute more extreme ratings to abnormal images (i.e. either appealing or unappealing) whereas normal images were more likely to evoke aesthetic ratings that were neither appealing nor unappealing, which is reflected in the interaction between stimulus and rating category. This trend in the data yielded an inter-subject response pattern, as we found no significant differences between subjects and rating category (ANOVA, F(1,14) = 1.26, p>0.27). This confirms an inter-subject response pattern displaying a tendency to attribute a binary rating strategy in abnormal conditions (i.e. appealing or unappealing responses) compared to normal conditions. Thus, this confirms that the binary rating strategy was not driven by a subset of subjects rating abnormal images with a higher frequency of appeal, while other subjects attribute a higher frequency of unappealing responses. Furthermore, the mean rating of normal and abnormal conditions for each subject show that subjects were fairly consistent with each other, although particular two subjects displayed extreme preference for either normal (subject#1) or abnormal (subject#7) conditions (fig. 2E). Despite the large standard deviations the stimuli displayed inter-subject consistency and individual stimuli pairs were not influenced significantly by normal or abnormal conditions (fig. 2F). Taken together, these results show a general response pattern across subjects to attribute a binary rating strategy suggesting that novelty/abnormality influence aesthetic ratings , .
We studied two main effects: [normal conditions>abnormal conditions] and [abnormal conditions>normal conditions]. In the first contrast we observed significant activations in the lateral occipital complex (LOC) bilaterally (fig. 3). We also found activation in the parahippocampal gyri bilaterally, and in the right inferior parietal lobule (Table 1). In the inverse main effect [abnormal conditions>normal conditions], significant activations were seen in the middle frontal gyrus bilaterally (MFG) and in the anterior cingulate gyrus, overlapping with the medial superior frontal gyrus (fig. 4). We also found activity in left temporo-parietal junction (TPJ), inferior parietal lobule (IPL) bilaterally and left retrosplenial cortex (Table 1).
Specific coordinates of activations are given in Table 1. Activations are surface rendered on the canonical SPM structural image. Bar plots show differences in parameter estimates between normal and abnormal conditions. AP = appealing; NE = neutral; NA = unappealing. Error bars indicate 90% confidence interval.
Significant voxels (p<0.001, uncorrected) were bilateral middle frontal gyrus (MFG), anterior cingulate (not shown), left temporo-parietal junction (TPJ), bilateral inferior parietal lobule (IPL) and retrosplenial cortex (not shown). Specific coordinates of activations are given in Table 1. Activations are surface rendered on the canonical SPM structural image. Bar plots show differences in parameter estimates between abnormal and normal conditions. AP = appealing; NE = neutral; NA = unappealing. Error bars indicate 90% confidence interval.
We next inspected interaction effects of context on aesthetic ratings to assess different brain areas involved in aesthetic judgment for normal and abnormal context. This analysis was motivated by the significant interaction between stimulus category and rating category observed in the behavioural data (see fig. 2B). Thus we performed the interaction [abnormal appealing>abnormal unappealing]>[normal appealing>normal unappealing]. The left temporal pole showed distinct specificity to a modulation of aesthetic ratings for abnormal conditions vs. normal conditions (fig. 5). The inverse interaction, revealed no suprathreshold activity (p<0.001, uncorrected; not shown).
Activation in left temporal pole (−46, 10, −36; k = 115, z = 3.11) significant at (p<0.001, uncorrected) is displayed on saggital and coronal sections of the canonical SPM structural image. On the right are displayed parameter estimates for voxels in left temporal pole. Bar plots show differences in parameter estimates between abnormal and normal conditions. AP = appealing; NE = neutral; NA = unappealing. Error bars indicate 90% confidence interval.
To further determine the effect of context on aesthetic ratings, linear parametric contrasts were used. We looked for voxels at which the BOLD signal showed a positive and a negative linear correlation with aesthetic ratings separate for normal and abnormal conditions.
For normal conditions, activity in the right posterior cingulate was positively correlated with aesthetic ratings, whilst bilateral lateral OFC showed a negative correlation (fig. 6). For abnormal conditions activity in right fronto-median cortex/frontal pole and bilateral inferior frontal gyrus was positively correlated with aesthetic ratings. Finally, right lateral OFC showed a negative correlation with aesthetic ratings for abnormal conditions (fig. 6).
Upper left panel: Activity in right posterior cingulate (6 −40 10; k = 139; z = 3.44) was positively correlated with aesthetic ratings in normal conditions. Upper right panel: Negative correlations with aesthetic ratings for normal conditions activated right lateral OFC (34 48 −14; k = 94; z = 3.59) and left lateral OFC (−34 46 −14; k = 48; z = 3.38). Lower left panel: Positive linear correlations with aesthetic ratings for abnormal conditions resulted in activity in left inferior frontal gyrus (−38 34 4; k = 347; z = 3.89), right inferior frontal gyrus (38 28 −4; k = 324; z = 3.25) and right frontomedian cortex/frontal pole (12 54 10; k = 124; z = 3.0). Lower right panel: Negative linear correlations with aesthetic ratings for abnormal conditions recruited right lateral orbitofrontal cortex (36 50 −14; k = 88; z = 3.54). All activation were found at p<0.001, uncorrected. Activations are overlaid on a saggital, coronal and axial sections of the canonical SPM structural image.
In order to identify areas involved in aesthetic evaluation irrespective of whether the viewed objects were in normal or abnormal contexts a conjunction analysis was applied. No regions were found to reach significance (p<0.0012, uncorrected). However when we dropped the level of significance (p<0.0052, uncorrected) we found that medial OFC showed a positive linear correlation with aesthetic ratings regardless of context (fig. 7A). This is consistent with previous studies which found that medial OFC correlates with beauty ratings in paintings , facial beauty ,  and pleasantness of sound . We therefore used small volume correction (SVC) constraining our analysis to this a priori region using the coordinates from a previous study . Activity reached corrected significance in this area (p<0.04, SVC) (fig. 7A).
(A) A conjunction analysis (p<0.0052, uncorrected) was performed as a linear positive correlation with aesthetic ratings for normal and abnormal conditions. Voxels were identified in the medial OFC (2, 50, –20; k = 15; z = 2.79). Using SVC this region reached a significant level (p< 0.04, SVC). On the right are shown parameter estimates for voxels in medial OFC for both conditions for each rating bin. (B) Voxels at which activity showed a negative correlation with subjective aesthetic ratings irrespective of stimulus condition resulted in activity in right lateral OFC (36, 50, −14; k = 84; z = 3.52). One the right are shown parameter estimates for lateral OFC. Activations are displayed on saggital sections of the canonical SPM structural image. Bar plots show differences in parameter estimates between abnormal and normal conditions. AP = appealing; NE = neutral; NA = unappealing. Error bars indicate 90% confidence interval.
We next looked for voxels at which the BOLD signal showed a negative linear correlation with aesthetic ratings regardless of context. This conjunction analysis revealed activity in right lateral OFC (p<0.0012, uncorrected) (fig. 7B). The differential activation in the medial and lateral aspects of the OFC as identified in the conjunction analysis, support previous reports indicating that abstract reward and punishment representations occur in the medial and lateral OFC respectively  with coordinates overlapping those found in the present study.
The aim of this study was to investigate the relationship between aesthetic judgment and objects in their normal contextual setting versus objects in abnormal contextual settings. In the results reported here we found a medial-lateral trend in OFC suggesting a general response mechanism to aesthetic judgment regardless of context (normal and abnormal). In addition the results from the parametric analysis suggest a differential activation pattern that guide aesthetic judgment of normal and abnormal conditions. Specifically normal conditions activate areas associated with semantic memory. Whereas areas involved in monitoring discrepancies between expectation and experience in memory encoding are activated in aesthetic judgment of abnormal conditions. We furthermore observed differences in neural activity when objects are viewed in normal contexts compared to when they are viewed in abnormal ones irrespective of aesthetic ratings. Specifically, the LOC and parahippocampal areas were significantly more active when objects were presented in normal context while frontal and parietal areas became engaged when the relationship between object and context was unexpected.
Contextual effects on aesthetic judgment
Although there were no significant effects of stimulus condition on average aesthetic ratings, there was a statistically significant tendency for subjects to use a binary rating strategy in abnormal conditions, i.e. rate stimuli as either appealing (20.7%) or unappealing (17%), whereas normal conditions received a significantly higher ratio of neutral response (22.5%). This response pattern was not significantly different across subjects, and may reflect a more careful evaluation before assigning an aesthetic value on novelty trials than normal trials , , which, although speculative, could indicate that subjects' inferred differential intentions to aesthetic judgments. Specifically, images of objects placed in an abnormal contextual settings lend themselves to aesthetic judgements (i.e. they are either appealing or unappealing), whereas objects placed within a normal contextual setting less obviously reflect aesthetic intention and are thus less likely to evoke aesthetic judgments (i.e. they are neither appealing nor unappealing). Such a differential employment of evaluation strategies would not necessarily be reflected in RTs in the present study, as subjects were instructed to view each stimulus passively (for 3500 ms) and to assign an aesthetic value to each image only after stimulus offset (see Material and Methods). Hence RTs might not have captured differential evaluation strategies. This view is substantiated by the failure to reach statistical significant differences in average RTs and in frequencies of each rating category for the two stimulus conditions. The interaction effect of contextual manipulation on aesthetic ratings resulted in a neural correlate in left temporal pole, a part of the brain thought to be implicated in semantic memory retrieval , . However, the temporal poles also have a putative role in social and emotional processing, since they are frequently found to be active during complex emotional tasks such as viewing emotional picture sets, a response that is absent in simpler emotional tasks, such as emotional face or gaze perception tasks . Several studies implicate the temporal pole in tasks that require subjects to analyse other peoples' emotions, intentions or beliefs –, including humour comprehension . Finally the temporal poles have been reported to be active in tasks that require explicit evaluation judgments such as emotional intensity , aesthetic judgments  and contextual framing that influence judgments of emotion in others . Consistent with these findings, our data suggest that the left temporal pole may be engaged during evaluation of affectively salient information and thus a differential evaluation strategy in abnormal than in normal trials. A likely strategy employed by subjects in abnormal trials may be an attempt to organize novel object-context pairings into a framework of prior knowledge and use this information to guide and bias aesthetic judgments . This may occur in a Baysian fashion by constantly updating and integrating information about the present context with previously stored knowledge , .
Context modulates different memory sub-systems in aesthetic judgment
Contextual effects on aesthetic judgment were further investigated in the linear parametric analysis where two areas were positively correlated with aesthetic ratings for abnormal conditions, namely bilateral inferior frontal gyrus, corresponding to BA 45, and another region where the center of activation was located within the fronto-median cortex/frontal pole (BA 9/10). The latter region has been related to introspective evaluation of internal mental states, i.e. one's own thoughts and feelings, and tasks that require self-reference . The former area has been implicated in oddball tasks . Specifically, activation of the inferior frontal gyrus may be involved in the decoding of stimuli, whereas the fronto-median cortex may modulate the aesthetic evaluation of each stimulus. In contrast, activity in the posterior cingulate cortex was positively correlated with aesthetic ratings in normal conditions. In opposition to abnormal conditions, this activation signifies a behavioral bias to use semantic or episodic memory to guide aesthetic rating where subjects may have attributed ratings according to how familiar they were. Indeed, activity in the posterior cingulate has been related to successful memory retrieval , .
Effects of aesthetic judgment irrespective of context
In the fMRI study of Kawabata & Zeki (2004), subjects viewed portrait, landscape, still life, and abstract paintings that they considered to be beautiful, compared to those that they considered neutral or ugly. Comparison of brain activity when viewing beautiful vs. ugly paintings yielded significant voxels in the medial OFC. The conjunction analysis in the present study revealed activation of medial OFC in a positive correlation with aesthetic ratings independent of stimulus conditions. The response of the medial OFC demonstrating sensitivity to the magnitude of aesthetic value is in accordance with studies on reward processing showing that the relative reward value of stimuli is reflected by the amplitude of neural activity in OFC –. The conjunction analysis further revealed activity in the right lateral OFC as a negative correlation with aesthetic ratings regardless of stimulus condition. This medial-lateral dissociation in the OFC has been observed in previous studies , . According to these studies, medial OFC is related to monitoring the reward value, whereas lateral OFC activity is related to the evaluation of punishers that can lead to a change in behaviour. In these studies the right lateral OFC is invariably implicated, but rarely the left. The results of our activation are consistent with this theory.
Contextual effects on object perception irrespective of aesthetic judgment
Objects viewed in abnormal contexts led to significantly more activity compared to normal contextual conditions in middle frontal gyrus bilaterally which forms part of the DLPFC, corresponding to BA 8, 9 and 46 and anterior cingulate cortex (ACC). This general cortical zone is activated whenever there is a departure from expectation. Examples include the viewing of objects dressed in colours not usually associated with them ; infrequent events –, visual and auditory oddball tasks , general perceptual and emotional deviance  and irregular temporal patterns . Previous functional imaging studies have implicated ACC in visual attention , emotional attention [49, and monitoring cognitive conflict . Our observation of increased ACC activation in abnormal compared to normal conditions is consistent with these proposed cognitive roles for ACC. Similarly, the TPJ has been associated with a role in detecting novel or otherwise salient stimuli , ,  in particular to targets appearing at unexpected locations . The results of the present study, taken together with the results of previous studies, suggest that the TPJ plays a general role in identifying novel or otherwise salient stimuli in the sensory environment that deviates from expectation.
Activity in the IPL presumably reflects the increased attentional demand caused by a violation of contextual expectations when objects are presented in abnormal contexts. Previous studies have shown that competing attentional subsystems work relative to each other in conflict-trials vs. non-conflict trials , . It has been suggested  that conflict-trials and selective attention involve distinct neural systems, which is consistent with the present findings.
The pattern of activation is significantly different when objects are presented in their normal contexts. In this case the activity did not involve frontal cortex but instead LOC bilaterally. Although we do not report activity in the LOC for the opposite contrast [abnormal>normal] the LOC was activated when objects were presented in abnormal context in comparison with an object-free baseline. We found that the LOC was significantly less active under abnormal than normal context. Generally, the LOC is a region that is believed to subserve object properties and shape analysis –. The activation of the LOC suggests that this region is not only sensitive to object properties but also to contextual information. Alternatively, the results could imply that elements related to the compositional structure of the original, i.e. normal conditions, may have resulted in a signal increase compared to the altered stimuli (i.e. abnormal conditions). This could be caused by irregularities in low-level visual features in abnormal conditions where a violation of basic image statistics occur more often compared to normal conditions.
Stimuli containing objects in normal contexts also elicited a stronger activation in bilateral parahippocampal gyri. Specifically, we observed a bilateral activation of the parahippocampal place area (PPA) , a region of cortex that has been found to respond selectively to houses, landscapes and other environmental sceneries. Our activation occupied a larger number of voxels on the right, which can be explained in light of previous studies showing that the right parahippocampal gyrus is most active during memory for landmarks , . It has however been suggested that the PPA, rather than being involved in processing scenes and landmarks specifically, is involved in processing contextual associations more generally –. Indeed, it has been shown that the PPA is sensitive to objects when presented with contextual information, whereas the retrosplenial cortex did not show this sensitivity, since the strength of activity was equivalent regardless of the presence of contextual information in this region . In our study we found activity in the retrosplenial cortex for the contrast [abnormal>normal]. Our results suggest, in accordance with previous studies , that there is a bifurcation of contextual information processing, with the PPA being significantly more sensitive to normal visual appearance, and the retrosplenial cortex responding relatively stronger to abnormal contextual processing. This suggest that contextual processing occurs in the retrosplenial cortex regardless of specific contextual properties, although we are unable to confirm this possibility as we did not present control conditions with isolated objects without context. Previous studies demonstrating parahippocampal activation during contextual associations have used behavioural tasks that required subjects to attribute explicit attention to context-related information in the stimuli used , . By contrast, in the present study subjects were instructed to rate each stimulus according to aesthetic value, which encourages subjects, although implicitly, to ignore context-related information. Despite these constrictions we observed significant activity in the cortical context network  such as the parahippocampal gyrus and the retrosplenial cortex ,  suggesting a robust response to contextual information in this network of areas. Critically, for the context contrasts we controlled for the distribution of aesthetic ratings such that the distribution of ratings were identical for both normal and abnormal conditions. Thus differences in aesthetic rating cannot account for the activations reported.
The author thanks Ollie Hulme, Stewart Shipp, Chris Frith and Martin Skov for helpful comments on the manuscript. This work was done under the supervision of Prof. S. Zeki.
Conceived and designed the experiments: UK. Performed the experiments: UK. Analyzed the data: UK. Wrote the paper: UK.
- 1. Biederman I (1972) Perceiving real-world scenes. Science 177: 77–80.I. Biederman1972Perceiving real-world scenes.Science1777780
- 2. Biederman I, Mezzanotte RJ, Rabinowitz JC (1982) Scene perception: Detecting and judging objects undergoing relational violations. Cognitive Psychology 14: 143–177.I. BiedermanRJ MezzanotteJC Rabinowitz1982Scene perception: Detecting and judging objects undergoing relational violations.Cognitive Psychology14143177
- 3. Boyce SJ, Pollatsek A, Rayner K (1989) Effect of background information on object identification. Journal of Experimental Psychology: Human Perception and Performance 15: 556–566.SJ BoyceA. PollatsekK. Rayner1989Effect of background information on object identification.Journal of Experimental Psychology: Human Perception and Performance15556566
- 4. Boyce SJ, Pollatsek A (1992) Identification of objects in scenes: The role of scene background in object naming. Journal of Experimental Psychology: Learning, Memory, and Cognition 18: 531–543.SJ BoyceA. Pollatsek1992Identification of objects in scenes: The role of scene background in object naming.Journal of Experimental Psychology: Learning, Memory, and Cognition18531543
- 5. Davenport JL, Potter MC (2004) Scene consistency in object and background perception. Psychological Science 15: 559–564.JL DavenportMC Potter2004Scene consistency in object and background perception.Psychological Science15559564
- 6. Friedman A (1979) Framing pictures: The role of knowledge in automatized encoding and memory for gist. Journal of Experimental Psychology: General 108: 316–355.A. Friedman1979Framing pictures: The role of knowledge in automatized encoding and memory for gist.Journal of Experimental Psychology: General108316355
- 7. Palmer SE (1975) The effects of contextual scenes on the identification of objects. Memory & Cognition 3: 519–526.SE Palmer1975The effects of contextual scenes on the identification of objects.Memory & Cognition3519526
- 8. Bar M (2004) Visual objects in context. Nat Rev Neurosci 5: 617–629.M. Bar2004Visual objects in context.Nat Rev Neurosci5617629
- 9. Fenske MJ, Aminoff E, Gronau N, Bar M (2006) Top-down facilitation of visual object recognition: object-based and context-based contributions. Progress in brain Research 155: 3–21.MJ FenskeE. AminoffN. GronauM. Bar2006Top-down facilitation of visual object recognition: object-based and context-based contributions.Progress in brain Research155321
- 10. Aminoff E, Gronau N, Bar M (2007) The parahippocampal cortex mediates spatial and non-spatial associations. Cereb. Cortex 27: 1493–1503.E. AminoffN. GronauM. Bar2007The parahippocampal cortex mediates spatial and non-spatial associations. Cereb.Cortex2714931503
- 11. Bar M, Aminoff E (2003) Cortical analysis of visual context. Neuron 38: 347–358.M. BarE. Aminoff2003Cortical analysis of visual context.Neuron38347358
- 12. Cox D, Meyers E, Sinha P (2004) Contextually evoked responses in human visual cortex. Science 304: 115–177.D. CoxE. MeyersP. Sinha2004Contextually evoked responses in human visual cortex.Science304115177
- 13. Goh JOS, Siong SC, Park D, Gutchess A, Hebrank A, et al. (2004) Cortical areas involved in object, background, and object-background processing revealed with functional magnetic resonance adaptation. J Neurosci 24: 10223–10228.JOS GohSC SiongD. ParkA. GutchessA. Hebrank2004Cortical areas involved in object, background, and object-background processing revealed with functional magnetic resonance adaptation.J Neurosci241022310228
- 14. Gronau N, Neta M, Bar M (2008) Integrated contextual representation for objects' identities and their locations. J Cogn Neurosci 20: 371–388.N. GronauM. NetaM. Bar2008Integrated contextual representation for objects' identities and their locations.J Cogn Neurosci20371388
- 15. Bar M (2007) The proactive brain: using analogies and associations to generate predictions. Trends in Cog Neurosci 11: 280–289.M. Bar2007The proactive brain: using analogies and associations to generate predictions.Trends in Cog Neurosci11280289
- 16. Berlyne DE (1970) Novelty, complexity and hedonic value. Perception and Psychophysics 8: 279–286.DE Berlyne1970Novelty, complexity and hedonic value.Perception and Psychophysics8279286
- 17. Berlyne DE (1974) Studies in the new experimental aesthetics. New York: Wiley. DE Berlyne1974Studies in the new experimental aestheticsNew YorkWiley
- 18. Oliva A, Torralba A (2007) The role of context in object recognition. Trends in Cogn Sci 11: 520–527.A. OlivaA. Torralba2007The role of context in object recognition.Trends in Cogn Sci11520527
- 19. Kawabata H, Zeki S (2004) Neural correlates of beauty. J Neurophysiol 91: 1699–1705.H. KawabataS. Zeki2004Neural correlates of beauty.J Neurophysiol9116991705
- 20. Zeki S, Marini L (1998) Three cortical stages of colour processing in the human brain. Brain 121: 1669–1685.S. ZekiL. Marini1998Three cortical stages of colour processing in the human brain.Brain12116691685
- 21. Stevens AA, Skudlarski P, Gatenby JC, Gore JC (2000) Event-related fMRI of auditory and visual oddball tasks. Magn Reson Imaging 18: 495–502.AA StevensP. SkudlarskiJC GatenbyJC Gore2000Event-related fMRI of auditory and visual oddball tasks.Magn Reson Imaging18495502
- 22. McCarthy G, Luby M, Gore J, Goldman-Rakic P (1997) Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI. J Neurophysiol 77: 1630–1634.G. McCarthyM. LubyJ. GoreP. Goldman-Rakic1997Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI.J Neurophysiol7716301634
- 23. Kirino E, Belger A, Goldman-Rakic , McCarthy G (2000) Prefrontal activation evoked by infrequent target and novel stimuli in a visual target detection task: an event-related functional magnetic resonance imaging study. J Neurosci 20: 6612–6618.E. KirinoA. BelgerGoldman-RakicG. McCarthy2000Prefrontal activation evoked by infrequent target and novel stimuli in a visual target detection task: an event-related functional magnetic resonance imaging study.J Neurosci2066126618
- 24. Clark VP, Fannon S, Lai S, Denson R, Bauer L (2000) Responses to rare visual target and distractor stimuli using event-related fMRI. J Neurophysiol 83: 3133–3139.VP ClarkS. FannonS. LaiR. DensonL. Bauer2000Responses to rare visual target and distractor stimuli using event-related fMRI.J Neurophysiol8331333139
- 25. Strange BA, Henson RNA, Friston KJ, Dolan RJ (2000) Brain mechanisms for detecting perceptual, semantic, and emotional deviance. Neuroimage 12: 425–433.BA StrangeRNA HensonKJ FristonRJ Dolan2000Brain mechanisms for detecting perceptual, semantic, and emotional deviance.Neuroimage12425433
- 26. Zeki S, Hulme OJ, Roulston B, Mueller R, Atiyah M (2008) The encoding of temporal irregular and regular visual patterns in the human brain. PLOS ONE 3: e2180.S. ZekiOJ HulmeB. RoulstonR. MuellerM. Atiyah2008The encoding of temporal irregular and regular visual patterns in the human brain.PLOS ONE3e2180
- 27. Friston KJ, Ashburner J, Frith CD, Poline JB, Heather JD, et al. (1995) Spatial registration and normalization of images. Hum. Brain Mapp 2: 165–189.KJ FristonJ. AshburnerCD FrithJB PolineJD Heather1995Spatial registration and normalization of images. Hum.Brain Mapp2165189
- 28. Friston KJ, Holmes AP, Worsley KJ (1999) How many subjects constitute a study. Neuroimage 10: 1–5.KJ FristonAP HolmesKJ Worsley1999How many subjects constitute a study.Neuroimage1015
- 29. Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston K, et al. (1996) A unified statistical approach for determining significant signals in images of cerebral activation. Hum. Brain Mapp 4: 58–73.KJ WorsleyS. MarrettP. NeelinAC VandalK. Friston1996A unified statistical approach for determining significant signals in images of cerebral activation. Hum.Brain Mapp45873
- 30. Aharon I, Etcoff N, Ariely D, Chabris CF, O'Connor E, et al. (2001) Beautiful faces have variable reward value: fMRI and behavioural evidence. Neuron 32: 537–551.I. AharonN. EtcoffD. ArielyCF ChabrisE. O'Connor2001Beautiful faces have variable reward value: fMRI and behavioural evidence.Neuron32537551
- 31. O'Doherty J, Winston J, Critchley H, Perrett D, Burt DM, et al. (2003) Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia 41: 147–155.J. O'DohertyJ. WinstonH. CritchleyD. PerrettDM Burt2003Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness.Neuropsychologia41147155
- 32. Blood AJ, Zatorre RJ, Bermudez P (1999) Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions. Nat Neuroscience 2: 382–387.AJ BloodRJ ZatorreP. Bermudez1999Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions.Nat Neuroscience2382387
- 33. O'Doherty JO, Kringelbach ML, Rolls ET, Hornak J, Andrews C (2001) Abstract reward and punishment representation in the human orbitofrontal cortex. Nat Neurosci 4: 95–102.JO O'DohertyML KringelbachET RollsJ. HornakC. Andrews2001Abstract reward and punishment representation in the human orbitofrontal cortex.Nat Neurosci495102
- 34. Damasio H, Grabowski TJ, Tranel D, Hichwa RD, Damasio AR (1996) A neural basis for lexical retrieval. Nature 380: 499–505.H. DamasioTJ GrabowskiD. TranelRD HichwaAR Damasio1996A neural basis for lexical retrieval.Nature380499505
- 35. Mummery CJ, Patterson K, Price CJ, Ashburner J, Frackowiak RS, et al. (2000) A voxel-based morphometry study of semantic dementia: relationship between temporal lobe atrophy and semantic memory. Ann Neurol 47: 36–45.CJ MummeryK. PattersonCJ PriceJ. AshburnerRS Frackowiak2000A voxel-based morphometry study of semantic dementia: relationship between temporal lobe atrophy and semantic memory.Ann Neurol473645
- 36. Olson IR, Plotzker A, Ezzyat Y (2007) The Enigmatic temporal pole: a review of findings on social and emotional processing. Brain 130: 1718–1731.IR OlsonA. PlotzkerY. Ezzyat2007The Enigmatic temporal pole: a review of findings on social and emotional processing.Brain13017181731
- 37. Castelli F, Happe F, Frith U, Frith C (2000) Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. Neuroimage 12: 314–325.F. CastelliF. HappeU. FrithC. Frith2000Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns.Neuroimage12314325
- 38. Gallagher HL, Happe F, Brunswick N, Fletcher PC, Frith U, et al. (2000) Reading the mind in cartoons and stories: an fMRI study of “theory of mind” in verbal and nonverbal tasks. Neuropsychologia 38: 11–21.HL GallagherF. HappeN. BrunswickPC FletcherU. Frith2000Reading the mind in cartoons and stories: an fMRI study of “theory of mind” in verbal and nonverbal tasks.Neuropsychologia381121
- 39. German TP, Niehaus JL, Roarty MP, Giesbrecht B, Miller MB (2004) Neural correlates of detecting pretense: automatic engagement of the intentional stance under covert conditions. J Cogn Neurosci 16: 1805–1817.TP GermanJL NiehausMP RoartyB. GiesbrechtMB Miller2004Neural correlates of detecting pretense: automatic engagement of the intentional stance under covert conditions.J Cogn Neurosci1618051817
- 40. Grezes J, Frith C, Passingham RE (2004) Brain mechanisms for inferring deceit in the actions of others. J Neurosci 24: 5500–5505.J. GrezesC. FrithRE Passingham2004Brain mechanisms for inferring deceit in the actions of others.J Neurosci2455005505
- 41. Mobbs D, Greicius MD, Abdel-Azim E, Menon V, Reiss AL (2003) Humor modulates the mesolimbic reward centers. Neuron 4: 1041–1048.D. MobbsMD GreiciusE. Abdel-AzimV. MenonAL Reiss2003Humor modulates the mesolimbic reward centers.Neuron410411048
- 42. Cunningham WA, Raye CL, Johnson MK (2004) Implicit and explicit evaluation: fMRI correlates of valence, emotional intensity, and control in the processing of attitudes. J Cogn Neurosci 16: 1717–1729.WA CunninghamCL RayeMK Johnson2004Implicit and explicit evaluation: fMRI correlates of valence, emotional intensity, and control in the processing of attitudes.J Cogn Neurosci1617171729
- 43. Jacobsen T, Schubotz RI, Höfel L, Cramon DY (2006) Brain correlates of aesthetic judgment of beauty. Neuroimage 29: 276–85.T. JacobsenRI SchubotzL. HöfelDY Cramon2006Brain correlates of aesthetic judgment of beauty.Neuroimage2927685
- 44. Mobbs D, Weiskopf N, Lau HC, Featherstone E, Dolan RJ, et al. (2006) The kuleshov effect: the influence of contextual framing on emotional attributions. Soc Cogn Affect Neurosci 1: 95–106.D. MobbsN. WeiskopfHC LauE. FeatherstoneRJ Dolan2006The kuleshov effect: the influence of contextual framing on emotional attributions.Soc Cogn Affect Neurosci195106
- 45. Fletcher P, Frith CH, Grasby PM, Shallice T, Frackowiak R, et al. (1995) Brain systems for encoding and retrieval of auditory-verbal memory. Brain 118: 401–416.P. FletcherCH FrithPM GrasbyT. ShalliceR. Frackowiak1995Brain systems for encoding and retrieval of auditory-verbal memory.Brain118401416
- 46. Cabeza R, Nyberg L (2000) Neural bases of learning and memory: functional neuroimaging evidence. Curr Opin Neurol 13: 415–421.R. CabezaL. Nyberg2000Neural bases of learning and memory: functional neuroimaging evidence.Curr Opin Neurol13415421
- 47. Kringelbach ML (2005) The human orbitofrontal cortex: linking reward to hedonic experience. Nat Re NeuroSci 6: 691–701.ML Kringelbach2005The human orbitofrontal cortex: linking reward to hedonic experience.Nat Re NeuroSci6691701
- 48. Nobre AC, Sebestyen GN, Gitelman DR, Mesulam MM, Frackowiak RSJ, et al. (1997) Functional localization of the system for visuospatial attention using positron emission tomography. Brain 120: 515–533.AC NobreGN SebestyenDR GitelmanMM MesulamRSJ Frackowiak1997Functional localization of the system for visuospatial attention using positron emission tomography.Brain120515533
- 49. Lane RD, Fink GR, Chau PM, Dolan RJ (1997) Neural activation during selective attention to subjective emotional responses. NeuroReport 8: 3969–3972.RD LaneGR FinkPM ChauRJ Dolan1997Neural activation during selective attention to subjective emotional responses.NeuroReport839693972
- 50. MacDonald AW, Cohen JD, Stenger VA, Carter CS (2000) Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288: 1835–1838.AW MacDonaldJD CohenVA StengerCS Carter2000Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control.Science28818351838
- 51. Downar J, Crawley AP, Mikulis DJ, Davis KD (2002) A cortical network sensitive to stimulus salience in a neutral behavioural context across multiple sensory modalities. J Neurophysiol 87: 615–620.J. DownarAP CrawleyDJ MikulisKD Davis2002A cortical network sensitive to stimulus salience in a neutral behavioural context across multiple sensory modalities.J Neurophysiol87615620
- 52. Knight RT, Scabini D, Woods DL, Clayworth CC (1989) Contributions of temporal-parietal junction to the human auditory P3. Brain Res 502: 109–116.RT KnightD. ScabiniDL WoodsCC Clayworth1989Contributions of temporal-parietal junction to the human auditory P3.Brain Res502109116
- 53. Corbetta M, Kincade JM, Ollinger JM, Mcavoy MP, Shulman GL (2000) Voluntary orienting is dissociated from target detection in human posterior parietal cortex. Nature Neurosci 3: 292–297.M. CorbettaJM KincadeJM OllingerMP McavoyGL Shulman2000Voluntary orienting is dissociated from target detection in human posterior parietal cortex.Nature Neurosci3292297
- 54. Shulman GL, Corbetta M, Buckner RL, Raichle ME, Fiez JA, et al. (1997) Top-down modulation of early sensory cortex. Cerebral Cortex 7: 193–206.GL ShulmanM. CorbettaRL BucknerME RaichleJA Fiez1997Top-down modulation of early sensory cortex.Cerebral Cortex7193206
- 55. Botvinick M, Nystrom LE, Fissell K, Carter CC, Cohen JD (1999) Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402: 179–181.M. BotvinickLE NystromK. FissellCC CarterJD Cohen1999Conflict monitoring versus selection-for-action in anterior cingulate cortex.Nature402179181
- 56. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, et al. (1999) Differential processing of objects in various viewing conditions in the human lateral occipital complex. Neuron 24: 187–203.K. Grill-SpectorT. KushnirS. EdelmanG. AvidanY. Itzchak1999Differential processing of objects in various viewing conditions in the human lateral occipital complex.Neuron24187203
- 57. Grill-Spector K (2003) The neural basis of object perception. Curr Opinion 13: 159–166.K. Grill-Spector2003The neural basis of object perception.Curr Opinion13159166
- 58. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, et al. (1995) Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA 92: 8135–8139.R. MalachJB ReppasRR BensonKK KwongH. Jiang1995Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex.Proc Natl Acad Sci USA9281358139
- 59. Epstein R, Kanwisher N (1998) A cortical representation of the local visual Environment. Nature 392: 598–601.R. EpsteinN. Kanwisher1998A cortical representation of the local visual Environment.Nature392598601
- 60. Maguire EA, Frith CD, Burgess N, Donnett JG, O'Keefe J (1998) Knowing where things are: parahippocampal involvement in encoding object locations in virtual large-scale space. J of Cognitive Neuroscience 10: 61–76.EA MaguireCD FrithN. BurgessJG DonnettJ. O'Keefe1998Knowing where things are: parahippocampal involvement in encoding object locations in virtual large-scale space.J of Cognitive Neuroscience106176
- 61. Mellet E, Bricogne S, Tzourio-Mazoyer N, Ghaem O, Petit L, et al. (2000) Neural correlates of topographical mental exploration. Neuroimage 12: 588–600.E. MelletS. BricogneN. Tzourio-MazoyerO. GhaemL. Petit2000Neural correlates of topographical mental exploration.Neuroimage12588600