Intangible features extraction in the processing of abstract concepts: Evidence from picture-word priming

Over the last decade, hypotheses ranging from linguistic symbol processing to embodiment have been formulated to account for the content and mechanisms responsible for the representation of abstract concepts. Results of recent studies have suggested that abstract concepts, just like concrete ones, can benefit from knowledge of real-world situational context, but that they can also be processed based on abstract pictures devoid of such situational features. This paper presents two semantic priming experiments to explore such mechanisms further. The first experiment replicates Kuipers, Jones, and Thierry (2018) in a cross-linguistic setting which shows that abstract concepts can be processed from abstract pictures devoid of tangible features. In the second experiment, we studied extraction mechanisms that come into play when participants are presented with abstract and concrete pictures that provide situational information to illustrate target abstract concepts. We expected this facilitatory effect to be limited to concrete picture primes. Our data analysed with both Bayesian and Frequentist tests showed however that even when presented with tangible situational information, the extraction of features still occurred for abstract pictures. We discuss the implications of this with respect to future avenues for studying the processing of abstract concepts.


Introduction
Abstract concepts have long been a riddle for the brain and cognitive sciences. The past decade saw a greatly enhanced research effort as regards abstract concepts representation (e.g., [1]; see [2] for a review). In spite of such efforts, challenging questions remain as to how abstract concepts convey meaning and the type of information in which their representation is encoded.

Dimensions of abstract concepts
With abstract concepts, there is no one-to-one matching between the concept-word and a single referent. Usually they refer to complex situations involving multiple objects and entities [3][4][5]. Until recently, their definition was formulated primarily in opposition to concrete concepts, therefore constraining the scope of the debate because the dichotomy between abstract instance, Wilson-Mendenhall, Martin, Simmons and Barsalou (2013) performed a task requiring deep conceptual processing of abstract concepts such as convince [32]. They observed neural activation patterns of non-linguistic brain areas associated with mentalizing and social cognition. In a more recent study, Harpainter et al. (2020) studied brain activation patterns for visual and motor abstract concepts such as beauty and fight [33]. In line with the grounding of representation framework, their results suggest some categories of abstract concepts benefit from a similar grounding mechanism to concrete concepts. Middle-ground theories, however, consider that both amodal and grounded content contribute to the representation of abstract concepts. For instance, Dove's representation pluralism ( [34][35][36] see also [2] and [37] for a review) represents just such a hybrid approach, according to which abstract concepts activate both linguistic and modality-specific features. Picture priming paradigms in the study of abstract concepts. The representation of abstract concepts in situations and contexts has been further investigated in recent years [25,29,30]. More recently, McRae, Nedjadrasul, Pau, Pui-Hei Lo and King (2018) presented abstract target words primed by pictures depicting scenes either related or unrelated to the target words [38]. For example, the concept discipline was primed by the picture of students lined up. When target words were preceded by related pictures, lexical decision latencies were shorter than when unrelated pictures were shown. Such results support the assumption of extraction of situational features in abstract concept processing.
In another recent study, Kuipers, Jones and Thierry (2018) used a similar priming paradigm, but with abstract picture primes, which lack tangible situational features [39]. The results showed shorter manual latencies and a smaller N400 amplitude in the EEG data for related abstract picture-word pairs than unrelated ones. The authors inferred that the abstract pictures conveyed the same meaning as the abstract words. The fact that abstract images devoid of situational context elicited an activation pattern similar to the abstract concepts they primed suggests the need for further investigation of the mechanisms involved in abstract concept processing.
The present study. After norming the abstract picture-word pairs from Kuipers et al. (2018) with French participants, we replicated their study in French in a first experiment to make sure their findings remained valid in a cross-linguistic setting. In a second experiment, we expanded on their study by comparing abstract picture primes to concrete ones within the same experiment [39]. For the first experiment, we expected to find the same facilitatory effect as in English, with shorter manual latencies for related abstract picture-word pairs than for unrelated ones. In a second experiment, we compared concrete picture primes allowing for situational grounding to abstract picture primes composed of intangible features. For this second experiment, in which concrete and abstract picture primes were presented together we also expected shorter latencies for related picture-word pairs than for unrelated ones in the concrete picture condition. However, and in line with the previously discussed role of situational features in the representation of abstract concepts, we expected this facilitatory effect to be limited to concrete picture primes, insofar as they allow for a more tangible form of situational or contextual information, and to disappear in respect of abstract picture primes (see [38]). Tangible features, as opposed to intangible ones, should allow for easier processing [40]. We expected that when presented with both types of features the participants would develop a strategy by allocating more resources to extracting tangible features, to the detriment of intangible ones.

Materials and methods
Participants. The same number of participants were tested in Experiment 1 as were tested by Kuipers et al. (2018) [39]. Twenty native French speakers from Université Clermont Auvergne, France, took part in this experiment (4 males; M age = 24, SD = 3). All participants were righthanded with corrected-to-normal vision. They all gave their informed written consent before taking part in the study. They were rewarded for their participation with a 10€ gift card. The study was approved by the local ethics committee (Comité d'éthique de la Recherche IRB-UCA).
Materials. The material used in Experiment 1 was crafted on the basis of the norming study described hereafter. For Experiment 1, the purpose was to create a set of semantically related and unrelated picture-word combinations based on the stimuli used in [39]. We obtained the abstract images and words collected for their study by contacting the authors. The materials consisted of 100 abstract images, each paired with one related and one unrelated abstract word. We translated the words into French and submitted the picture-word combinations to a group of 166 French participants (56 men, M age = 21.7, SD = 4.6) using Qualtrics (2020) in a preliminary study. As in Kuipers et al. (2018), participants were asked, "How strongly does the word below match the above picture?" (on a scale of 0 to 10) [39]. We kept only the 71 picture-word combinations that elicited the highest scores for the related conditions, and those that elicited the lowest scores for the unrelated conditions (M Related = 6. Procedure. Experiment 1 used exactly the same procedure as Kuipers et al. (2018), except that we recorded only reaction times, and not EEG. We presented abstract image-word combinations that were either semantically related or unrelated (see Fig 1 for example stimuli) [39]. Both the related and unrelated modalities were presented in a within-subjects design. Stimuli were presented using E-prime software v2.1 on a 15-inch Dell PC colour monitor with a refresh rate of 60 ms and connected to an AZERTY keyboard. Participants were seated at a distance of 50 cm from the monitor. Each trial began with a fixation cross, displayed in the middle of the screen for 300ms followed by an abstract picture that remained on the screen for 5000ms. The fixation cross was then replaced with an abstract target word, which stayed on the screen until the participant answered, or until 5000ms had elapsed. The participant's task was to indicate, using the D and K keys on the AZERTY keyboard, whether the target word was semantically related or unrelated to the previous picture. Response latencies were recorded, as well as accuracy. The experiment took approximately 20 minutes to complete.

Results
Reaction times. Latencies > 3 SDs above or below each participant's mean latencies for each condition were excluded from the analyses (less than 2% of the data). Mean correct latencies were analysed by means of a one-sided paired-samples t-test testing the same directional hypothesis as Kuipers et al., (2018). We conducted analyses within a Bayesian framework relying on prior specification of theoretical knowledge which includes effect sizes and the direction of hypotheses [33], see also [41]. The analysis revealed an effect of Target Word Type with shorter latencies for the related pairs than the unrelated ones (M Related = 1411 ms; SD = 429; M Non_Related = 1529 ms; SD = 464; Mean Difference = 118 ms; 95CI [21,214]; t(19) = 2.55, p = 0.01, Cohen's d = 0.57, see Fig 2). We conducted a Bayesian paired-samples t-test using JASP [42]. For the informed prior we used Oosterwijk's recommendation (t-distribution with location 0.350, scale 0.102, and 3 degrees of freedom), because it is considered a good prior for small-to-medium effect sizes (see [43]). This analysis showed a Bayes Factor of 10.93 (median = 0.39; 95CI [0.19-0.67]). This means the results are about 11 times more likely under the alternative hypothesis compared to the null. Taken together, the Bayesian analyses show substantial to strong support for the finding that abstract images can elicit a semantic priming effect [44].
Yet, this effect is somewhat surprising and rather counter-intuitive with respect to the role of situational grounding in concept representation. We therefore conducted supplementary analyses in a Bayesian replication framework to test for the robustness of such an effect. To that end, we applied a Replication Test in Rstudio [45] (ReplicationBF package [46]) which can be used to assess the success or failure of a replication attempt by testing whether the effect identified in previous studies is present or absent in the replication attempt [46], whether the effect size in the replication is equal to that found in the original study [47], and whether the effect is present when the data are pooled in a meta-analysis [48]. The replication test relies on sample sizes and t-values and requires no specification of a prior distribution, thus escaping the influence of a researcher's choice of priors, but it uses the posterior distribution of the https://doi.org/10.1371/journal.pone.0251448.g001 original study as a prior for the replication attempt ( [49,50] but see [51]). The results of the replication tests are reported in the supplementary analyses. For the equality-of-effect size Bayes factor, support for the null hypothesis, according to which there is no difference between effect sizes, is indicative of a successful replication. The tests provided substantial support for the equality of effect sizes (Equality B01 � 5; see Table A.1 in S1 Appendix), with the fixedeffect meta-analysis Bayes factor test showing strong support for the presence of an overall effect (Meta B10 � 30). Finally, the new Bayes factor for replication shown in Figure A.1 in S1 Appendix indicates that the results are about 12.5 times more likely under the proponent's hypothesis that the effect is consistent with the one found in the original study as opposed to the sceptic's hypothesis that the effect is spurious [46]. Errors. A paired-samples t-test showed no effect of Target Word Types for the errors (t (19) = 0.14; p = 0.90 ns).

Discussion
The aim of Experiment 1 was first to extend the replicability of Kuipers et al. (2018) to another language before considering expanding it in a follow-up experiment [39]. As in the study by Kuipers and colleagues, we obtained shorter latencies for related combinations than for unrelated ones. They interpreted their results in terms of meaning processing and concluded that it was possible that abstract pictures can convey the same meaning as lexical abstract concepts. Given the counter-intuitive results obtained by Kuipers et al. (2018), especially given the important role of situational information, we decided to conduct Bayesian analyses [46], the results of which provided strong evidence in favour of Kuipers and colleagues' alternative hypothesis, according to which abstract pictures can convey the same meaning as abstract concepts. Furthermore, we conducted a supplementary replication test to ensure our attempt at replication was successful. We demonstrated therefore that the effect of picture-word priming using abstract pictures could be replicated in a cross-linguistic setting using different abstract images. Despite our analyses confirming the evidence uncovered by Kuipers and colleagues we do not rule out other possible interpretations (see General Discussion). Having confirmed the replicability of a study using such stimuli, we are now keen to expand Experiment 1 in a paradigm that would allow abstract concepts to be extracted from situational pictures. Experiment 2 aimed to implement such a paradigm. To that end, we kept the abstract priming pictures from Experiment 1 and added a condition for which participants were presented with concrete priming pictures that provided situational information.

Materials and methods
Participants. An independent group of 90 students from Université Clermont Auvergne who met the same criteria as for Experiment 1 took part in Experiment 2 (18 males; M age = 20.2; SD = 2.86). They all gave their informed written consent before taking part in the study. The study was approved by the local ethics committee (Comité d'éthique de la Recherche IRB-UCA).
Materials. The materials used for Experiment 2 were the result of a second norming study. The purpose of the norming study for Experiment 2 was to find corresponding concrete images for each picture-word combination in Experiment 1. To that end, we searched online databases for images likely to elicit the same meaning as the abstract words. For each word, we selected two potential images which we then submitted to an independent group of 146 French participants (38 men, M age = 22.4, SD = 5.6) using Qualtrics. Participants were asked, "How strongly does the word below match the above picture?" (on a scale of 0 to 10). Picture-word combinations that obtained a low score (< 5 on a scale of 0 to 10) for both image options were discarded. The remaining 56 stimuli combinations (abstract/concrete image-related/unrelated word) were used in Experiment 2 (see Fig 1 for an example of stimuli).
Procedure. We used a similar procedure as for Experiment 1, but expanded it with 2 within-subjects variables: Priming Image Type (abstract vs. concrete) and Target Word Type (related vs. unrelated). The stimuli were divided in a Latin square with 4 modalities: abstract prime images, concrete prime images, semantically-related target words and unrelated target words. Following this Latin-square design, the pictures were divided into two lists so that each picture presented in the related condition to participant 1 was also presented to participant 2 in the unrelated condition. Participants were exposed to all variables but saw only one of the 4 possible types of picture-word combination for each stimulus (see Fig 1 for an illustration of the trial procedure with an example of stimuli).

Results
Reaction times. Latencies > 3 SDs above or below each participant's mean latencies for each condition were excluded from the analyses (i.e.~2% of the total data). Data from 3 participants were discarded based on z-scores that showed very slow RTs in all conditions (zscores > 2.99). The following analyses are based on the data from 87 participants.
Mean correct latencies are presented in Fig 2. Mean correct latencies were analysed with a 2 (Priming Image Type = Concrete vs. Abstract) � 2(Target Word Type = Related vs. Unrelated) repeated measures ANOVA.
This analysis revealed a main effect of Priming Image Type with shorter latencies for concrete images compared to abstract images ( 2). The semantic facilitatory effect was therefore present for both types of prime stimuli. A Bayesian repeated measures ANOVA compared four models to the null model. We kept the default JAPS prior for fixed effects (r scale prior width = 0.5; [47,48]). Based on Bayes Factors, the model including only the types of priming images was 5770 times more likely than the null model. There is therefore strong evidence for the type of prime model. There was also strong evidence for the model including only the type of word target (BF = 1.902x10 13 ), for the model including both main effects (BF = 2.990x10 18 ), and for the interaction model (BF = 2.551x10 25 ). Finally, we compared the main effects model to the interaction model (2.551x10 25 /2.990x10 18 = 8.533 x10 6 ; based on the Baws Factor suggested by Mathôt, 2017 [52]) and obtained strong evidence for the interaction model.
We used a Bayesian paired-samples t-test to explore this interaction, with an informed prior following Oosterwijk's recommendation. The analysis showed a Bayes Factor of 7.67 (median = 0.30; CI [0.13, 0.45]) in favour of the alternative hypothesis for the difference between related and unrelated target words in the abstract picture primes condition, and a Bayes Factor of 4.42x10 15 (median = 1.19; CI [0.89, 1.48]) in favour of the alternative hypothesis for the difference between related and unrelated target words in the concrete picture primes. These Bayesian analyses concur with the ones obtained from traditional paired-samples t-test and confirm substantial evidence for the effect of the abstract picture primes and extreme evidence for the concrete ones.
Errors. Given the difficulty of the task, especially concerning the abstract image condition, we expected larger rates of errors overall, but more specifically for the abstract image primes.
Indeed, a repeated measures ANOVA revealed a main effect of Priming Image Type with a higher percentage of errors for the abstract images compared to the concrete images (M Abstract_Image = 29%; SD = 16.5; M Concrete_Image = 12%; SD = 10; F(1,87) = 189.87, p < 0.001, η p 2 = 0.69). As for Experiment 1, the results showed no main effect of Target Word Type.

Discussion
Experiment 2 expanded on Experiment 1 by comparing abstract to concrete pictures priming abstract lexical concepts. We opted for a semantic, as opposed to a lexical decision task given that Recchia and Jones (2012) suggested the latter did not yield deep semantic processing and therefore accounted for the discrepancies between results in the abstract concept processing literature [53]. Recognising this might be a concern, however, we checked for such surface features based on analyses of emotional valence in an additional study. We presented the pictures to an independent pool of 49 participants online and asked them to rate the emotional valence of the pictures on a scale of 0-10. We then used this valence variable in a post-hoc analysis and found it failed to explain the variance in latencies for semantic priming (based on a repeated measures ANOVA (F(2, 48) = 1.76; p = .19 ns)). Based on this analysis we can rule out a shallow picture-word association based only on surface physical features.
This facilitatory effect was significantly stronger for concrete pictures than for abstract ones. However, it was weaker in relation to Experiment 1, although it did not disappear for abstract pictures.
This result expands on those obtained by Kuipers et al. (2018), showing that abstract concepts can be processed on the basis of abstract and concrete pictures alike and more broadly from tangible and intangible features. The Bayesian models provide decisive evidence in favour of the alternative hypothesis of semantic priming. The error rates obtained were quite high compared to traditional effects in the semantic priming literature. We were not surprised by these rates, however, as the task was more difficult in comparison to traditional priming studies. We asked Kuipers et al. (2018) for the error rates for their study (27% error rates for the related and 31% for the unrelated items), which confirmed our intuition that such stimuli would elicit higher error rates compared to more traditional semantic priming studies.

General discussion
The aim of the present studies was to investigate conceptual processing mechanisms based on picture-word combinations in terms of the role of situational features compared to intangible abstract features.
In the first experiment and in line with Kuipers and colleagues, we investigated conceptual processing mechanisms in relation to abstract pictures. The results showed participants were able to process concepts from abstract pictures devoid of tangible features (therefore replicating Kuipers et al., 2018 results and extending them to the French language). To the best of our knowledge, Kuipers et al. (2018) were the first to provide evidence that abstract pictures can activate the meaning of abstract concepts. The data of Experiment 1 which was analysed using a Bayesian replication framework, corroborated their findings. This fact alone broadens the scope of conceptual representation beyond the usual debate of linguistic vs. situational features (see for e.g., [14,25,34]). It also goes to show that abstract concepts are even richer and activate features beyond their linguistic or situational components [53].
In a second experiment, we gave participants an opportunity to extract situational features (see also [38]). Results showed a stronger facilitatory effect for concrete picture-words pairs compared to abstract ones. Participants relied more heavily on the priming pictures with distinguishable features. Our results support previous findings according to which some types of abstract concepts are grounded in situations and events [25,32,40].
From the evidence from the two experiments, it appears that is possible to represent abstract concepts based on abstract and intangible pictures as well as on concrete and tangible ones. McRae et al. (2018) showed that in a picture-priming paradigm similar to our "concrete images" condition, participants could process the meaning of abstract concepts derived from pictures depicting scenes. Kuipers et al. (2018) showed how this processing could also be derived from abstract pictures that differ from the stimuli used by McRae and colleagues because they are devoid of tangible features. In the present study, and based on the assumption that it is costlier to extract intangible features when tangible ones are available, we showed that extraction mechanisms still occurred in the case of abstract pictures.
The immediate interpretation of these results makes reference to how Kuipers and colleagues and McRae and colleagues interpreted meaning processing based on tangible and intangible features. This initial interpretation infers that there is a semantic level of processing and implies a featural view of semantic representation according to which concepts can be broken up into a set of defining features that represent their meaning (see [54] for an account of the featural view). If confirmed, this assumption would mean such features can be other than lexical or situational, namely also abstract and intangible.
For instance, Bolognesi and Vernillo (2019) proposed 'Abstraction by Metonymy' as a novel grounding mechanism for abstract concepts in the pictorial mode, where they use verbo-pictorial metaphors to investigate people's ability to illustrate abstract concepts [55]. According to their hypothesis, this abstraction process allows for an inferential mechanism that moves from concretely depicted entities to more abstract ones. In the present studies, the reverse mechanism seems to have occurred. It might be that in the abstract image condition, participants extracted intangible features and inferred a more tangible representation of the concept. Therefore, these results could be construed as complementing the Abstraction by Metonymy.
In conclusion, by directly comparing the two types of features for representation, we were able to show that abstract concepts can be processed based on situational features and abstract ones. For us, these conclusions signal the need to explore further the mechanisms of concept representation and abstraction.