Conceived and designed the experiments: MH JR JB SB RL. Performed the experiments: MH CM. Analyzed the data: MH DA. Contributed reagents/materials/analysis tools: MH JR DA. Wrote the paper: MH JR.
The authors have declared that no competing interests exist.
Negotiation and trade typically require a mutual interaction while simultaneously resting in uncertainty which decision the partner ultimately will make at the end of the process. Assessing already during the negotiation in which direction one's counterpart tends would provide a tremendous advantage. Recently, neuroimaging techniques combined with multivariate pattern classification of the acquired data have made it possible to discriminate subjective states of mind on the basis of their neuronal activation signature. However, to enable an online-assessment of the participant's mind state both approaches need to be extended to a real-time technique. By combining real-time functional magnetic resonance imaging (fMRI) and online pattern classification techniques, we show that it is possible to predict human behavior during social interaction
Neuroscientific studies of the brain mechanisms of social decision-making offer new insight which helps to incorporate human behavior into economic models. In the framework of neuroeconomics, cognitive and neural constraints of the complex processes of social decision-making are explored
Using a real-time noninvasive technique based on fMRI, we investigated the neural correlates of social decision-making and tried to already infer the decisions made by participants involved in social interaction from brain activation during scanning. We employed a well-established economic game called the ultimatum game (UG), in which two players split a given amount of money. One player acts as the proposer, retaining one share of the money and offering the remaining share to the other player (the responder). The responder can either accept or reject the proposer's offer. If the offer is accepted, the money is split as proposed. If the offer is rejected, neither player receives anything. According to the notion of profit maximization, the proposer is expected to offer the smallest possible sum of money and the responder to accept this offer, because even the smallest profit is preferable to no monetary reward
Social interaction as in the ultimatum game may lead to conflicts between players' goals and internal attitudes and social norms, which elicit emotions. These conflicts require considerable cognitive effort to be resolved
However, the statistical analysis used in these studies relies on the comparison of mean blood oxygen level dependent (BOLD) signals calculated from many trials, leaving the question open whether these effects are strong enough to be reliably detected in
Here our goal was to discriminate complex brain states occurring in social interactions on the basis of the BOLD signal in a small number of distinct brain regions in real time. Including only few relevant brain areas allowed us to adapt the model parameters of a Relevance Vector Machine (RVM) classifier
Ten healthy male subjects (23–28 years, mean: 24.7±1.6 years) with normal or corrected to normal vision were examined after providing written informed consent. The experiments were approved by the local ethics committee of the Medical Faculty of the University of Magdeburg. One subject was excluded from the study after reporting doubts about whether he was playing with human partners. Data from two subjects served for the initial training of the classifier that was subsequently used to examine seven subjects. To avoid cross-gender effects, only male volunteers participated in the study
At the beginning of a session, participants met two male individuals, who were introduced to them as the proposers in the UG. Participants were told that the actual proposer would be chosen randomly from these two individuals for each single trial and that proposers do not interact with each other during the experiment. This procedure was chosen because personal contact between responder and proposer is considered to be an essential prerequisite to establishing a social bond between players
Brain activity was measured and analyzed using rtfMRI and real-time pattern classification while each volunteer completed 60 trials of 22 s length each. In each trial the amount to be split was shown for 2 s. Subsequently, the offer was shown to the volunteer for 12s. The BOLD signal of the first 10 s after showing the offer was used to predict the upcoming decision. During the following response phase of 4 s length, participants pressed one of the two buttons to convey their decision. Finally, the payoff in the current trial was presented for 4 s and the next trial started immediately (see
Stimuli were backprojected with an LCD beamer onto a transparent screen. Subjects had to press buttons with their left or right index finger to convey their decisions on the given offers. The mapping between buttons and responses (for either accepting or rejecting) was switched randomly for each trial and displayed at the beginning of each response phase. This prevented the classifiers from using brain activity related to preparation of motor responses
The blood oxygen level dependent (BOLD) response was measured in a 3 Tesla whole-body MRI scanner equipped with Avanto gradient system (Siemens Medical Systems, Erlangen, Germany). The imaging protocol consisted of a gradient echo EPI sequence for BOLD imaging with repetition time (TR) of 2 s, time to echo (TE) of 29 ms, and a flip angle of 90°. Thirty-one slices with axial slice orientation covering the whole brain were acquired. The matrix size was 64×64 and spatial resolution was 3.4×3.4×4 mm3.
The vendor's EPI BOLD sequence (system version VA25A) and the corresponding image reconstruction programs were modified to export each EPI volume immediately after acquisition and internal motion correction to the host computer of the MR scanner (see
The components highlighted in gray depict the vendor-specific measurement system (Siemens Trio with SYNGO Version VA25A). Initially, the original MR data are fourier-transformed and motion-corrected by the vendor image processing unit (Image PC). The reconstructed data are then transferred to the host computer (External PC). There the data are processed using custom software (rtExplorer). This software performs pre-processing, statistics, online classification, and documentation of the classification results. The participants' responses are processed in the stimulus PC and transferred to the external PC for evaluation of the classification and for retraining the classifier during the ongoing session.
The locations of the regions of interest (ROIs) used in the online procedures were pre-specified on the basis of functional MRI data from preliminary experiments including two participants (120 trials) using the same experimental paradigm. The results of a whole-brain offline trained Support Vector Machine (SVM) classifier indicated signal changes predictive of the volunteers' decisions in anterior insula, lateral prefrontal cortex, and occipital cortex (see also
Three distinct brain regions were used for classifying the volunteers' decisions: anterior insula (AI), lateral prefrontal cortex (LPFC) and occipital cortex (OC). See
Brain Region | Center Coordinates [mm] | Volume [mm3] | ||
x | y | z | ||
|
||||
left | −50 | 28 | 11 | 3798 |
right | 50 | 28 | 11 | 3798 |
|
||||
left | −38.5 | 20 | −1 | 2925 |
right | 38.5 | 20 | −1 | 2925 |
|
0 | −88 | 3 | 15606 |
MNI coordinates for the centre points of the regions used for the computation of the t-values. The ROI volume is in mm3.
These preliminary data sets were also used to obtain an initial solution for the model parameters of the real-time classifier used in the online experiment. This allowed us to start prediction without first acquiring an exhaustive set of individual data. Importantly, using only a small set of ROIs reduced the feature space sufficiently allowing us to continuously adapt the classifier in real time by retraining with newly arriving individual data.
In the online experiments, custom rtfMRI analysis software was used to process the incoming image data as soon as they were acquired
The three t-values per trial served as input for the online classifier, a nonlinear Relevance Vector Machine Classifier
We refer to
During the experiment, the initial training set (
The classifier was continuously retrained in each trial using the expanded training set. As such, the system adapted the model parameters based on subject-specific activation states in real time and included these in the forecast of volunteers' future decisions to improve classification accuracy.
The RVM applied in online prediction makes use of Bayesian inference to obtain sparse solutions for classification. By computing a posterior distribution, it provides probabilistic classification and has the same functional form as the well-known Support Vector Machines:
Here
To test the reliability of the online prediction, we determined individual empirical guessing levels to ensure that the online discrimination rates were not obtained by pure guessing but exploit information inherent to the data. The theoretical guessing level of a two-class experiment (e.g. accept or reject an offer) is 50% (perfect coin toss). However, other factors, such as the relative frequencies of the two classes in the training set, may influence the classifiers' strategy and bias the guessing level to much higher values than expected
We estimated individual empirical guessing levels by permuting the decision vectors in each subject's data set. Permutation destroys the coherence between the observed BOLD data and volunteers' decisions but retains other information such as class size ratio. The classifier was then retrained, and all trials were classified according to the new training set. These steps were repeated 500 times to estimate the mean guessing level and the 95% confidence interval. Empirical guessing levels were calculated as the geometric mean of the guessing levels for the classes accept and reject
Additional offline classification was performed to assess classification performance achievable using BOLD data from the whole brain and to further investigate the neural correlates of the decision process. Preprocessing included motion-correction, spatial smoothing with a 9 mm Gaussian kernel, and linear detrending. Furthermore, low-frequency signal fluctuations were removed using a high-pass filter with a cut-off frequency of 0.01 Hz, and BOLD volumes were normalized to 3×3×3 mm3 MNI space. Non-brain voxels were excluded by applying a MNI brain template. Before combining the BOLD-data over subjects we first z-scored every subject's data individually. This normalization was done voxel-wise and as a result the BOLD-time series of each voxel had a mean of 0 and a standard deviation of 1. The volumes of the 2nd, 3rd, and 4th scan after the presentation of the offer were averaged for every subject. This resulted in 420 average functional brain volumes serving as single samples for whole brain classification. Our learning algorithm thus provides a cross-subject model based on single trial data. We then used this to classify the single trial data of the single subject excluded from the classifier training.
The 2nd, 3rd, and 4th volumes after offer presentation were chosen because the participants reported in the post-scanning questionnaire that they made their internal decisions quickly (i.e. always in less than 5 seconds) after an offer was revealed and always before the accept/reject screen was shown. We thereby also avoided including information about the actual motor response, because in the interval included the participants did not know the mapping of the two buttons for accepting or rejecting the offer.
We used feature selection, a very common approach in pattern classification, to reduce the number of features (voxels) in the input space. This was done on a training set by correlating signal changes with the volunteers' two different decisions. Voxels with correlation values between −0.15 to 0.15 were excluded. Since we wanted to analyze which voxels the trained classifier judged as informative we chose this relatively liberal value to somewhat reduce the number of voxels used for classification without being overly restrictive. Approximately 104 voxels were retained for subsequent classification using this criterion.
For offline classification, we used a publicly available implementation of a SVM
Theoretical and empirical guessing levels were determined analogous to the approach in real-time prediction, in a permutation test with 500 repetitions.
We extracted the spatial patterns used by the classifier to discriminate between different brain states from the weight vector
The percentages of acceptance for the five types of offers are depicted in
Values are calculated as rate of accepted offers over seven volunteers. Labels on the x-axis show the split rate: (proposer: responder).
As depicted in
The arrows mark the empirical guessing levels.
To assess the gain in correct predictions achieved by continuously retraining the classifier, we simulated the online procedure both with and without retraining. The overall prediction accuracy increased by 10.7% when novel data were used to retrain the classifier showing a clear benefit of retraining with individual data (
The number of additional correct predictions using individual data acquired during the experiment in a sliding window of six trials are shown. Each window includes 42 single predictions (6 trials times 7 subjects).
In addition to binary classification accuracy, RVM classification provides a continuous posterior probability estimate for each classified decision. The mean probability estimates for the five types of offers are depicted in
Means and standard deviations plotted were calculated over the seven volunteers tested in online analysis. The labels on the x-axis depict the split rate: (proposer: responder).
The analysis of the activation of the signal variation immediately following an offer showed a clear difference between frontal and posterior ROIs. Higher BOLD signal in AI and LPFC predicted rejection, whereas a higher BOLD signal in OC predicted acceptance of an offer (
Differences were calculated between 1st to 2nd and 3rd to 5th scan after the offer and averaged over the seven participants. Bold signal in AI (slope linear fit 0.062, p<0.05) and LPFC (slope linear fit 0.11, p<0.05). In contrast, signal decreases in OC when the likelihood of acceptance decreases (slope linear fit −0.16, p<0.05).
In an additional offline analysis, we pooled the single trial fMRI data from all but one subject (leave on subject out) to train classifiers and test generalization among subjects. This improved the correct classification rate greatly to an average of 81.2%. The average guessing level of the offline classification determined in permutation tests was 51.1%±2.3% SD (average 2.5% and 97.5% quantiles were 47.3% and 55.1%, respectively). Again, the correct classification rate clearly exceeds the 95% confidence interval for guessing. This results clearly shows that there is information about rejection or acceptance of a decision in the BOLD data that is similar among participants. Moreover, this analysis allowed us to derive brain areas informative about a participant's decision from a larger set of subjects and to validate the choice of the ROIs in the online experiment.
Brain Region | Center Coordinates [mm] | Volume [mm3] | ||
x | y | z | ||
|
4 | 58 | 6 | 1629 |
|
−2 | 32 | −4 | 999 |
|
||||
Right | 20 | 58 | −8 | 783 |
|
||||
Right | 36 | 24 | 2 | 1278 |
|
||||
Left | −45 | 28 | 28 | 405 |
Right | 40 | 32 | 22 | 2115 |
|
2 | 12 | −4 | 1278 |
|
||||
Left | −55 | −42 | 27 | 1089 |
Right | 54 | −46 | 26 | 1215 |
|
−2 | −90 | 5 | 4941 |
|
||||
Left | −47 | −70 | −35 | 1476 |
Right | 50 | −72 | −32 | 1521 |
MNI coordinates of discriminating volumes found with linear SVM in the offline procedure. Weight values were thresholded at P<0.05 and minimum cluster volume was 300 mm3. STS: superior temporal sulcus.
The decision process we investigated so far includes at least two sub-processes: one related to the evaluation of the offer (e.g. low or high earning) and another related to the choice of the response (reject or accept an offer). We analyzed our data according to choices in the previous offline analysis. However, since choice and offer value are correlated over the full scale of offers it is possible that BOLD activity related to evaluation of offer value is more predictive about the subjects' UG responses than choice related BOLD activity, at least on the full scale of offers. To investigate this hypothesis each trial received two labels: one for the offer (low or high) and one for the choice (accepted or rejected) and we trained two classifiers with trials of the same dataset sorted in the two different ways (choice or value). The datasets used for classifier training have to be balanced with respect to each of the four possible label combinations (low/accept, high/accept, low/reject, and high/reject) to avoid unwanted classifier bias. In order to maximize the number of trials available in the four label combinations we distinguish high from low offers around the categorical decision border between 80∶20 and 70∶30 split rates where acceptance rate sharply drops. We labeled 50∶50, 65∶35, and 70∶30 trials as high offers and 80∶20 and 90∶10 trials as low offers. The combination reject/high offer contained the lowest number of samples (n = 19), restricting the number of trials used in the other three combinations in the training of the classifier. In order to avoid selection bias, we evaluated classifier performance on 200 balanced subsets of 76 samples each of which included the 19 rejected/high offers and 19 samples randomly drawn from each of the other three label combinations. The average LOOCV classification accuracy revealed that it was possible to discriminate high from low offers on the basis of the single trial BOLD activity (65.9% correct ±6.2% SD) with some success. On the contrary, discrimination according to choice (accept/reject) was around chance level (56.4% correct ±5.9% SD). This result indicates that brain processes related to the evaluation of offer value rather than the choice related activation allows the prediction of the subject's response on the wide range of offer values used in the offline prediction.
Although no systematic brain activation difference related to choice (reject/accept) may exist over a wide range of offer values, this does not rule out, that a strong link between choice and brain activity exists that may manifest in a predictable and restricted regions along the offer scale where a large change in choices (accept/reject) is found. In the following analysis we aim to demonstrate such an isomorphism between brain activity and behavior for choice related activity. The reasoning behind this analysis follows previous work from us and other groups
We tested this prediction by training an SVM in an LOOCV to discriminate between adjacent offers. Therefore, we repeatedly (200 times) selected 42 examples from each split rate. The number of trials used per repetition was limited by the class with the lowest number of examples, in this case the number of trials in the 50∶50 split rate. In concordance with our hypothesis we found the highest discrimination rate between trials from 70∶30 and 80∶20 splits (71.4%±5.53% SD). The single trial discrimination rate was at guessing level for the comparisons among trials between split rates 80∶20 vs. 90∶10 (53.4%±5.3% SD), and 65∶35 vs. 70∶30 (54.6%±4.7% SD), and moderate for the discrimination between split rates 50∶50 vs. 65∶35 (65.9%±5.2% SD). It is important to note that this pattern of results cannot be explained by value differences between offers. The 70∶20 offer differs by 10% (or 0.3 Eurocent) from the 80∶20 offer, the same amount the 80∶20 differs from the 90∶10 and even less than the 65∶35 differs from the 70∶30, and the 50∶50 from the 60∶35 offer (
Brain Region | Center Coordinates [mm] | Volume [mm3] | ||
x | y | z | ||
|
||||
|
||||
Left | −41 | 49 | −12 | 1404 |
Right | 38 | 47 | −12 | 845 |
|
||||
Right | 38 | 25 | 1 | 1836 |
|
||||
Left | −45 | −49 | 44 | 459 |
|
6 | 14 | −10 | 1080 |
|
||||
Left | −43 | −76 | −36 | 702 |
|
||||
|
3 | 54 | 2 | 3240 |
|
−6 | 28 | −12 | 1485 |
|
||||
Right | 23 | 55 | −12 | 2160 |
|
||||
Right | 39 | 23 | 30 | 999 |
|
||||
Left | −51 | −49 | 42 | 2214 |
|
−4 | −93 | 1 | 2835 |
|
||||
Left | −43 | −73 | −35 | 918 |
MNI coordinates of discriminating volumes for classification between offers 70∶30 vs. 80∶20 and 50∶50 vs. 65∶35. Weight values were thresholded at p<0.05. Minimum cluster volume was 300 mm3.
In this study, we show that it is possible to predict the behavior of social agents acting as responders in the UG in real time using BOLD measurements of brain activity to detect complex emotional and cognitive states. Offline analyses confirmed the ROIs selected for online prediction on two pilot subjects and the rejection rates. More detailed analyses of the information about split rate and decision outcome available in the BOLD-data strongly supports the notion that brain activity related to expected subjective value of an offer rather than choice predict the subjects behavior over a large range of offer values. the mere decision process. Importantly, we find that information about choice in the BOLD activity predicts the behaviorally observed categorical change from offer acceptance to rejection.
We found that AI and LPFC are both predictive of the rejection of an offer on a trial-by-trial basis, in the online as well as in the offline analysis. Both brain areas are involved in emotion regulation and adjustment during social interaction
As opposed to AI and LPFC, activation in early visual cortex decreased with unfavorable split rates. It has been shown that attention strongly influences the responses of cortical neurons
In sum, the results from the online experiment suggest that activation in brain areas reflecting the subject's emotional and motivational state and self-regulatory processes can be used to discriminate accepted from rejected offers.
When playing against a computer that is creating offers in a random order, it makes no sense to reject an offer from an economic perspective. Thus, the participants' best strategy to optimize monetary gain would have been to accept any offer. However, responders in our study rejected unfair offers (20% of 3 euros and less) significantly more often than fair offers. This is the behavior expected in the repeated version of the UG (
Thus, in the ultimatum game the acceptance of an offer is correlated with the expectation of a financial incentive but, in addition, hedonic states following costly punishment of an unfair offer may also contribute to adjustment of behavior
Unlike other offline “mind reading” approaches (compare e.g.
Moreover, RVM provides posterior probabilities for single trial class membership, which can be useful in classification-based neurofeedback (compare
Whether a responder in the UG finally decides to reject or accept a specific offer depends on a multitude of internal factors. Among these factors are emotions such as the feeling of being treated fairly as well as rational considerations of reward maximization. The extraction of this information about the way a social agent is tending with a decision in real time
In sum, our results show that, in single trials, it is possible to reliably predict acceptance or rejection of an offer from BOLD measurements of brain activity before the subject reveals the decision with an overt response. However, more detailed analyses indicated that prediction of the decision was based on brain processes related to the perception and evaluation of the offer rather than processes related to the decision itself. Importantly, AI, VS, and LOFC, brain areas related to emotional self-regulation and reward processing for adjustment of behavior, appeared to be strong determinants of overt behavior in the ultimatum game. The decisions derived from the activation in these brain areas paralleled the behaviorally observed categorical transition from high likelihood of acceptance to high likelihood of rejection of an offer when the split rate fell below 70∶30. The framework presented here can be used in future studies to augment information available in social interaction with information about current brain states that remain hidden in traditional approaches.
(TIF)
(TIF)
(TIF)
(DOC)
We thank Claus Tempelmann for his helpful support with the fMRI experiments.