Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The psycholinguistic and affective structure of words conveying pain

  • Eleonora Borelli ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft

    eleonora.borelli@unimore.it

    Affiliations Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy, Center for Neuroscience and Neurotechnology, University of Modena and Reggio Emilia, Modena, Italy

  • Davide Crepaldi,

    Roles Data curation, Formal analysis, Visualization, Writing – review & editing

    Affiliations Department of Cognitive Neuroscience, International School for Advanced Studies (SISSA), Trieste, Italy, Milan Center for Neuroscience (NeuroMi), Department of Neurology and Neuroscience, University of Milano-Bicocca, Milan, Italy

  • Carlo Adolfo Porro,

    Roles Conceptualization, Funding acquisition, Methodology, Writing – review & editing

    Affiliations Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy, Center for Neuroscience and Neurotechnology, University of Modena and Reggio Emilia, Modena, Italy

  • Cristina Cacciari

    Roles Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Validation, Writing – review & editing

    Affiliations Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy, Center for Neuroscience and Neurotechnology, University of Modena and Reggio Emilia, Modena, Italy

The psycholinguistic and affective structure of words conveying pain

  • Eleonora Borelli, 
  • Davide Crepaldi, 
  • Carlo Adolfo Porro, 
  • Cristina Cacciari
PLOS
x

Abstract

Despite the flourishing research on the relationships between affect and language, the characteristics of pain-related words, a specific type of negative words, have never been systematically investigated from a psycholinguistic and emotional perspective, despite their psychological relevance. This study offers psycholinguistic, affective, and pain-related norms for words expressing physical and social pain. This may provide a useful tool for the selection of stimulus materials in future studies on negative emotions and/or pain. We explored the relationships between psycholinguistic, affective, and pain-related properties of 512 Italian words (nouns, adjectives, and verbs) conveying physical and social pain by asking 1020 Italian participants to provide ratings of Familiarity, Age of Acquisition, Imageability, Concreteness, Context Availability, Valence, Arousal, Pain-Relatedness, Intensity, and Unpleasantness. We also collected data concerning Length, Written Frequency (Subtlex-IT), N-Size, Orthographic Levenshtein Distance 20, Neighbor Mean Frequency, and Neighbor Maximum Frequency of each word. Interestingly, the words expressing social pain were rated as more negative, arousing, pain-related, and conveying more intense and unpleasant experiences than the words conveying physical pain.

Introduction

May words be painful? Undoubtedly yes and in several respects, as literary sources, personal experience, and a handful of recent behavioral and brain-imaging studies have shown (e.g., [13]). Words represent the main tool for describing the physical and social experience of pain (e.g., [45]) and can be metaphorically extended to characterize social phenomena, as exemplified by the title of a recent article in Science: “Growing pains for global monitoring of societal events” [6].

Notwithstanding the pervasiveness and relevance of the words used to convey pain at different levels (henceforth pain words), the psycholinguistic and affective characteristics of this important part of the lexico-semantic domain of negative words have never been specifically tested. Norms about affectively-laden words already exist for a variety of languages, including Italian (e.g., [78]), but due to the general aim of these datasets they contain only a limited number of pain words (we return on this point below). This study was devised to bridge this gap creating a normed corpus of Italian pain-related words (Words of Pain database, henceforth WOP). WOP may at the same time contribute to the literature on the characteristics of affectively-laden words and provide a tool for experimental studies of pain.

Language is more than a mere medium when it comes to share our pain experiences. In fact, it has been shown that processing pain-related words is associated with enhanced activation of part of the neural circuitry underlying physical pain experiences [1,2,9,10]. Medical studies also have observed that the presentation of pain words can modulate the perception of noxious stimuli, especially in chronic pain patients [3]. The mechanisms underlying these important effects of pain words are still under investigation. It has been suggested that the comprehension of pain words may occur via an embodied simulation involving reliving and/or retrieving pain-related information (e.g. [11]), in analogy to what happens in the empathic response to pain (e.g., [12]). In fact, merely observing, thinking about, or inferring that someone else is in pain have been shown to trigger the emergence of physical pain [13], a phenomenon known as synesthesia for pain [1416]. A wealth of studies on empathy for pain has led to suggest the existence of common neural substrates that map the perception of pain in oneself and in the others (for an overview, see [13]).

Describing pain in medical settings

We use linguistic stimuli to convey our own experience of pain since early childhood [17]. From a medical viewpoint, assessing the sensory, affective, and cognitive impact of the pain experience to the sufferers still represents a challenge [18,19]. “Pain is defined and ultimately evaluated by subjective report. Much can be inferred from objective measures of anatomy, physiology, and behavior, but verbal report remains the standard by which all other measures are compared” ([19], p.1309). In fact, medical doctors typically categorize the pain of sufferers primarily “translating” their pain reports into a finite set of descriptors that are thought to “capture and categorize facets of the pain experience as evidenced in the endorsement and ranking of pain descriptors” ([19], p. 1387). These descriptors are contained in pain questionnaires devised to assess different types of pain. For instance, in one of the sections of the McGill Pain Questionnaire (MPQ, [20]; for an overview see [18,19]), the pain sufferer is asked to indicate what his/her present pain feels like choosing among 78 descriptors (e.g., fearful, itching, hot-burning). According to Melzack (1975), these pain descriptors reflect three distinct components of pain that be divided in “sensory descriptors” that convey the sensory qualities of pain (e.g., burning), “affective descriptors” that convey the emotional components of pain (e.g., punishing), and “evaluative descriptors” that provide a global evaluation of the pain experience (e.g., unbearable). However, since the MPQ was primarily designed by clinical doctors (as all the other pain questionnaires), the verbal items were not controlled for any of the psycholinguistic and emotional variables that are known to modulate the cognitive demands of their processing.

Pain words and the affective lexicon

Pain words are part of the general domain of affectively-laden words. Consensus exists about the fact that the affective space is best characterized by a two-dimensional structure formed by two orthogonal dimensions that together account for most of the variation in how affective stimuli are evaluated [2123]. Valence ranges from positive to neutral to negative and is thought to reflect the general motivational significance of a stimulus. Arousal ranges from low to high and is thought to reflect the degree to which a stimulus prepares a person for action or captures and focus attention [21,24]. Most current models of affective word processing assume that valence and arousal are orthogonal variables ([25,26]; for an overview of consistent and inconsistent results, see [2729]).

In general, affectively-laden words (and sentences) are processed faster and more efficiently, elicit larger electrophysiological responses since very early processing stages and activate affect-related brain regions (e.g., medial PFC, ACC, insula, and amygdala) more strongly than affectively-neutral linguistic stimuli (for overviews, see [3032]). That affective connotations facilitate processing may reflect the grounding of these word meanings in bodily emotional experiences [33,34].

A wealth of studies has shown that negatively valenced information is associated with more complex mental representations that require a more demanding cognitive processing than positively valenced information (Negativity bias, [35,36]). Unpleasant events or stimuli, compared to matched pleasant ones, evoke larger emotional responses, longer duration responses with a broader impact on the cognitive system. According to the Automatic Vigilance Hypothesis, humans preferentially attend to negative stimuli and this attention to negative Valence diverts processing resources away from other stimulus properties, leading to longer response times [3742]. Indeed, negative words typically elicit slower color naming [43], lexical decisions ([3944] but see [45] for the mitigating role of arousal), and word naming [46] than neutral and/or positive words. This would reflect the fact that survival primarily depends on our ability to withdrawing from negative events and scenario [47]. Since the withdrawal-aversive system has a processing priority over the approach-appetitive system [48], negative stimuli recruit more attentional resources than positive stimuli. This hypothesis has been supported by word studies using different tasks [46,4951]. However, recent experiments have questioned this negative emotion processing advantage showing that once the non-emotional characteristics of words (e.g., length, frequency, and orthographic neighborhood) were considered, and neutral control words were used as well, much of the processing difference between negative and positive words disappeared ([44] but see [52]). In some cases, the asymmetry was even reversed with a processing advantage for both positive and negative words over neutral words [29,47,53]. Then, in an ERP study, Hofmann et al. [45], showed that lexical decision responses were speeded at a similar extent for positive and high-arousal negative words suggesting that the level of arousal differently interacts with positive and negative valences in early lexical processing.

Physical pain and social pain

According to the International Association for the Study of Pain (IASP), physical pain is defined as the unpleasant sensory and emotional experience associated with actual or potential tissue damage or described in terms of such damage. Physical pain is often associated with a noxious physical stimulus. However, painful experiences are triggered not only by noxious stimuli but also by events, feelings, and thoughts that usually lead individuals to experience a form of pain that recently has been defined as social pain [5,54] (although it incorporates also aspects of a more general feeling of pain not necessarily associated to social events). Social pain is thought to derive from social exclusion, rejection, loss and grief (e.g., [55,56]) and generally is described as intense as actual, physical pain [57].

Across languages we extend the use of physical pain words to describe experiences of social pain (e.g., broken heart, soul scar) (e.g., [5,58]). This use can be epitomized by the words of Hillary Clinton in her first speech after 2016 US election defeat, “This is very painful and will be for a long time” [59]. There is now growing consensus that the use of physical pain words to describe social pain is more than just a convenient metaphor. In fact, several brain-imaging studies have shown that the painful feelings following social pain rely on some of the same neural regions sub serving physical pain processing (e.g., [54,55], but see [60,61]). Notwithstanding the fact that social pain is mostly expressed using physical pain words, the stimuli of many behavioral and brain-imaging studies on social pain were not words but rather other type of visual stimuli (e.g., pictures, the Cyberball paradigm; for overviews see [60,61]).

Why creating word corpora?

Many studies investigating human cognition use tasks that require verbal stimuli as experimental material because words can be tightly controlled for their attributes [62]. Therefore, using stimuli controlled for the psycholinguistic and affective variables that are known to affect the time it takes to encode a word has become crucial. This has led to the growth of large-scale studies in different languages aimed at creating databases providing normative information about the most important variables affecting lexico-semantic processing (e.g., English Lexicon Project, [63]; French Lexicon Project, [64]; Dutch Lexicon Project, [65]). Typically, these normative data are obtained from rating and/or reaction times studies in which participants evaluate these variables and/or perform word recognition tasks. These large-scale studies produce databases offering psycholinguistic, affective, and behavioral measures rated by large numbers of participants (e.g., [6669]). Other databases provide normative data about specific set of words or specific psycholinguistic, semantic, and/or affective characteristics of the stimuli (e.g., affective words [70,71,7,8]; nouns [72]; monosyllabic words [73]; idiomatic expressions [74]; semantic categories [75,76]). Italian databases providing psycholinguistic, semantic, and/or general affective normative about sets of Italian words are available as well (e.g., [7,8,7684]). However, none of them is specifically focused on pain words, nor they include a number of pain-related items to make them suitable for pain experiments.

For many years, research on emotion has predominantly used the Affective Norms for English Words (ANEW, [85]). ANEW provides a set of normative data about the valence, arousal, and dominance of 1,034 American English words. Language-specific adaptations of the ANEW are now available for many languages including Italian [7,8], Brazilian Portuguese [86], Chinese [87], Dutch [70,71,88], European Portuguese [89], Finnish [90], French [91], and Spanish [92]. Other datasets on affective words have been proposed (e.g., [69,9395]), some of which also provide ratings of lexico-semantic variables and/or lexical decision times for larger set of stimuli (e.g., [96]). Concerning Italian, Montefinese et al. [7] and Fairfield et al. [8] collected ratings for psycholinguistic and affective variables of 1,121 Italian words (extending the original ANEW) respectively from younger and older adults. Due to the general aims of these databases, only a few of the words we use to convey pain were included. For instance, the 1121 words tested in Montefinese et al [7] only included 76 of the pain words of WOP. More importantly, WOP differs from these databases in that it offers not only the psycholinguistic and affective characteristics of 512 words, but also ratings related to pain-related variables (see below) relevant to the research on pain.

The present study

In this study, we selected 512 Italian pain words including (1) nouns referring to objects, conditions, events, and feeling that may cause physical pain (e.g., ago, needle; malattia, illness) or social pain (e.g., abbandono, abandon; lutto, grief); (2) adjectives that describe physical or social pain (e.g., atroce, dreadful), painful objects (e.g., appuntito, pointed), and painful events and moods (e.g., deprimente, depressing; inconsolabile, inconsolable) and adjectives that convey sensory as well as emotional aspects of pain (e.g., addominale, abdominal; diffuso, radiating as well as costrittivo, constrictive; fastidioso, uncomfortable); (3) verbs referring to pain, painful objects, and actions that may be painful or cause pain (e.g., bruciare, to burn; sbattere, to stab). For each of these words, we collected ratings concerning psycholinguistic (Familiarity, Age of Acquisition, Imageability, Concreteness, Context Availability) and affective properties (Valence, Arousal). We also tested how much each of these 512 words is associated to pain (Pain-relatedness) and how intense and unpleasant is the pain experience conveyed by their meaning (Pain Intensity and Pain Unpleasantness, respectively). According to the experimental literature on pain, Intensity taps on the sensory-discriminative dimension of pain (i.e., the physical characteristics of the noxious stimulus, namely how intense is the pain) and Unpleasantness taps on the affective-motivational dimensions of pain (i.e., its emotional characteristics, namely how much disturbing is the pain) [97]. In addition, we collected data concerning the Length, Written Frequency, N-Size, Orthographic Levenshtein Distance 20, Neighbor Mean Frequency, and Neighbor Maximum Frequency of each word.

We also analyzed the three word classes (i.e., nouns, adjectives and verbs) separately since there is evidence that word class affects the timing and characteristics of affective word processing (e.g., [32,98100]. This could reflect the fact that, as Palazova et al. [100] pointed out, adjectives that typically describe characteristics, states, and traits may have a more direct link with emotions than verbs, that typically describe actions or events, and then nouns, that denote more or less concrete objects. Finally, we analyzed the psycholinguistic, affective and pain-related differences between physical and social pain words.

Materials and methods

Participants

1020 undergraduates, PhD students, postdocs, and senior researchers (276 male and 744 female; age range: 18–40, mean age: 24.2 years, SD = 4.3) of the Universities of Parma, Modena and Reggio Emilia volunteered to participate in this online study. They were all Italian native speakers. Participants were recruited through an e-mail sent to the specific mailing lists of these Universities. The study was performed in accordance with the ethical standards of the 2013 Declaration of Helsinki and was approved by the Departmental Ethics Committee of the International Advanced Studies Institute, SISSA.

Materials

The stimulus set consisted of 512 Italian words associated to pain. To select the words, we used an extraction procedure typical of the computational linguistic research. This procedure assumes that the lexicon is a metrical space in which words are separated by distances that depend on the degree of semantic similarity between words measured through their statistical co-occurrence distribution in texts [101]. We used the word dolore (pain) as an anchor point and selected the content words co-occurring with it in a window of 25 words to the left and 25 words to the right of dolore in a corpus of Italian newspapers’ texts (La Repubblica Corpus, [78]) as well as medical dictionaries, blogs, and pain questionnaires. The resulting word list was formed by: a) 199 nouns (in their singular form), 46 of which referred to social pain; b) 218 adjectives (in the singular masculine form), 15 of which referred to social pain; c) 75 verbs (in the infinite form), nine of which referred to social pain; d) 20 words that may belong to different classes depending on context (e.g., cieco, blind; estremo, extreme, can either be nouns or adjectives), one of which referred to social pain (e.g., intimo, intimate).

Since 48 out of the 512 words could be used to refer to both physical and social pain (e.g., aborto, abortion; commozione, sentiment/concussion), we asked 67 different participants (24 male and 43 female; age range: 19–40, mean age: 33 years, SD = 5.1) to decide whether each of these 48 words predominantly referred to physical or social pain. The percentages of choice are listed in the database. The database resulting from this selection procedure contains a lower number of words referring to social pain than to physical pain. This may reflect the fact that many of the words referring to physical pain are metaphorically extended to convey social pain as well.

Tested variables

We tested the following variables:

  1. (1). Familiarity, i.e., the frequency with which a word occurs in everyday life [102]. The rating scale went from one (not at all familiar) to seven (extremely familiar);
  2. (2). Age of Acquisition (AoA), i.e., the age at which a word was learnt [103]. The rating scale went from one (0–2 years) to seven (13 and older) with intervening points spanning two years [104]. It has been shown that AoA represents a reasonable estimate of the actual age at which a word is acquired. In fact, AoA ratings significantly correlate with more objective measures of word acquisition age (e.g., [105108]);
  3. (3). Imageability, i.e., the ease with which a word gives rise to a mental image [109,110]. The rating scale went from one (not at all imaginable) to seven (extremely imaginable);
  4. (4). Concreteness, i.e., the degree to which a word refers to a perceptible entity [111,112]. The rating scale went from one (not at all concrete) to seven (extremely concrete);
  5. (5). Context Availability, i.e., the ease with which a word may call to mind a context or circumstance [113]. The rating scale went from one (context not at all available) to seven (context extremely available). Although we may be more able to call to mind a context for familiar than for unfamiliar words, it has been shown that Context Availability and Familiarity tap on different aspects of language processing [114];
  6. (6). Valence, i.e., the degree to which a stimulus is perceived as emotionally negative or positive [22]. The rating scale went from -3 (extremely negative) to +3 (extremely positive) through 0 (neither negative nor positive) [70,71] to keep a more intuitive negative to positive scale [115];
  7. (7). Arousal, i.e., the excitation potential of a stimulus regardless of whether it is positive or negative [116]. The rating scale went from one (not at all arousing) to seven (extremely arousing);
  8. (8). Pain-relatedness, i.e., the extent to which the word was associated to pain. The rating scale went from one (not at all associated) to seven (extremely associated);
  9. (9). Pain Intensity, i.e., the intensity of the pain conveyed by the word meaning. This variable was rated using a Visual Analogue Scales (VAS) [117], in analogy to the way in which it is measured in the experimental pain literature; the VAS consisted of a line of 10 cm with extremes labeled as Not at all intense and Extremely intense;
  10. (10). Pain Unpleasantness, i.e., the unpleasantness of the pain conveyed by the word meaning. As per Pain Intensity, this variable was rated using a Visual Analogue Scales (VAS) [117], in analogy to the way in which it is measured in the experimental pain literature; the VAS consisted of a line of 10 cm with extremes labeled as Not at all unpleasant and Extremely unpleasant.

When the meaning of a word was unknown, subjects were instructed to choose the option "I don’t know this word".

Familiarity was always rated first since past research has shown that having previously seen a word could affect Familiarity ratings [118]. The variables were presented in the same order in all the questionnaires.

In addition, we collected the following data:

  1. (11). Word Length, measured as number of letters;
  2. (12). Word frequency (Zipf), according to the Subtlex-IT corpus [80], a database of Italian word frequencies based on 130 million words extracted from film and television subtitles;
  3. (13). Neighborhood Size (Nsize), namely, the number of words of the same length differing from the target word by exactly one letter [119];
  4. (14). Orthographic Levenshtein Distance 20 (OLD20), namely, the mean edit distance to the 20 closest neighbors. We collected this measure since Yarkoni et al. [120] identified it as a better indicator of lexical density than the Nsize;
  5. (15). Neighbor Max Frequency, namely, the frequency of the most frequent orthographic neighbor, according to the Subtlex-IT corpus [80];
  6. (16). Neighbor Mean Frequency, namely, the mean frequency of the orthographic neighbors, according to the Subtlex-IT corpus [80].

Procedure

Participants received an e-mail asking whether they were willing to participate in a web survey. The e-mail also contained instructions on how to access a randomly assigned, self-paced questionnaire via a web site. The 512 stimuli were randomly distributed over twenty Google Form questionnaires each composed by 24 to 26 words (Table 1). Each questionnaire started with an introduction that explained that the aim of the study was to collect information about the words we use to describe pain in its broadest sense and specified the time approximately necessary to complete the questionnaire (45 minutes). Then the questionnaire contained questions concerning demographic information (i.e., gender, age, mother tongue, and education), and whether the responder suffered or had ever suffered of any forms of chronic pain or intense and repeated migraines. To reduce unpredictable effects of random word orders, the same word list was repeated for each of the ten variables of interest. Written instructions were presented at the beginning of each rating scale. They contained a definition of the variable to be rated, an explanation on how to use the Likert (or VAS) scale, and two examples of words rated with extreme values. The original Italian instructions and their English translation can be found in S1 Text.

Open access policy

The WOP database, in an Excel format including both raw and standardized data, is available on the web at https://figshare.com/s/188257a8c7de933ba28a.

Statistical analyses were carried out using R 3.4.0 [121] and IBM SPSS Statistics 24.0 [122].

Results and discussion

Analyses of the demographic characteristics of participants (Fig 1 and Table 2) showed no significant differences in the gender of the responders to the twenty questionnaires [F (1,19) = 1.553, p = .061, η2 = .029]. A significant difference instead emerged in the mean age of the responders [F (1,19) = 1.858, p = .014, η2 = .034]. Specifically, Tukey’s HSD post hoc test revealed a significant age difference of the responders to questionnaires 5 and 20 (M = 22, SD = 2.8 and M = 25.7, SD = 5.7, respectively).

thumbnail
Fig 1. Demographic characteristics of participants.

Age distribution across the 20 questionnaires, each represented by a different line. The grey scale for each line represents the gender proportion in the specific sample of participants.

https://doi.org/10.1371/journal.pone.0199658.g001

thumbnail
Table 2. Descriptive statistics for the variables considered in this study.

https://doi.org/10.1371/journal.pone.0199658.t002

Scores were standardized within subjects using a z-transformation. Because score mean and variance changed substantially across participants, and because each participant only received a subset of the stimuli, this metric was necessary for directly comparing the ratings between subjects.

Missing responses/omissions were 1.68% of the dataset. Most of these missing responses (94.63%) came from participants who reported that they did not know a given word. Unknown words could be due to the presence of a few stimuli belonging to the medical jargon (e.g., urente, burning; cefalico, cephalic). The mean percentage of response “I don’t know the word” was similar across the different variables suggesting that, in general, when participants did not know a word, they did not rate it further. Occasionally participants were able to rate only some of the variables (notably Familiarity and AoA) for words they have heard but whose exact meaning they were not sure about. The overall number of valid data points after excluding missing responses/omissions was 257,518.

Data were cleaned of uninformative/misleading data points in two steps. First, for the variables rated on 7–point scales, we excluded data points coming from participants who showed little or no variance in their responses since they had always used only one or two values of the rating scale. This procedure was applied separately for each variable and led to the exclusion of 2.58% of the data points (ranging from 0.4% for AoA to 8.8% for Familiarity). Similarly, we controlled if participants had zero variance in the Intensity and Unpleasantness ratings, meaning that likely they did not rate the words at all, leaving the cursor in the starting position. This led to the exclusion of the ratings of two participants for the Intensity scale (.19% of the available valid data points) and 11 participants for the Unpleasantness scale (1.09% of the available valid data points).

The second step allowed identifying outliers through the procedure illustrated in Rodriguez and Laio [123]. According to this procedure, participants are modeled as points in an N–dimensional space, where N equals the number of words that each participant rated. The ratings for each word define the position of each participant/point in this space, so that participants with similar judgments will be close and participants with different judgments will be relatively far apart (see data in S1 Fig). Rodriguez and Laio’s procedure was applied separately for each questionnaire and variable and led to the further exclusion of 2.72% of the remaining data points overall (ranging from .94% for Context Availability to 3.98% for Intensity). The final number of valid data points at this stage was 243,824, evenly distributed across the 10 variables of interest (Fig 2). Table 2 provides descriptive statistics of the final dataset.

thumbnail
Fig 2. Distribution of valid data points.

Distribution of the final number of valid data points (243,824) across the 10 variables of interest.

https://doi.org/10.1371/journal.pone.0199658.g002

We also compared the ratings obtained in the present study with those of the study on the affective lexicon of Montefinese et al. [7] for the 76 words and the variables shared by the two datasets (i.e., Familiarity, Imageability, Concreteness, Valence, Arousal). All correlations were significant (Table 3). This further suggests that our norms can be confidently used for word selection in affective word studies.

Table 2 contains untransformed values for all the words together, as well as separately for each word class, and for physical and social pain.

Reliability of the measures

We computed the reliability of the data for each variable by calculating the average split–half correlation over 1,000 random replicates, separately for each of the 20 questionnaires. Overall, the results showed a very strong reliability of the measures (Table 4 and Fig 3). The mean correlation value of each variable was very high, ranging from a minimum of r = .87 for Context Availability to a maximum of r = .98 for AoA. The mean correlation value of all the variables was M = .94 (SD = .03) suggesting that the collected ratings are highly reliable. Context Availability fared a little worse than the other variables, perhaps because it depends heavily on experience that is likely to vary quite substantially across participants. Because scores were standardized within participants, they are all reported on the same scale (z scores). Most variables had a rather symmetrical distribution, reasonably well centered on their mean and median (Fig 4). This was particularly true for Concreteness, Valence, Arousal, Pain-relatedness, and Intensity. Familiarity was quite left-skewed instead, not surprisingly given that the database includes several stimuli belonging to a medical jargon that may be rather unfamiliar to many participants. In addition, we cannot exclude that this result may also reflect the tendency to feel more familiar with pro-social and benevolent communication (Linguistic positivity bias, [7,94,124126]). Overall, all the variables seemed quite well suited to investigate their effects on behavior with enough statistical power across their entire distribution.

thumbnail
Table 4. Correlation values for each variable resulting from the average split–half correlation for each questionnaire.

https://doi.org/10.1371/journal.pone.0199658.t004

thumbnail
Fig 3. Measures reliability.

Distribution Boxplots of the overall half-split reliability distributions over 1,000 random replicates, run separately for each questionnaire and for each variable.

https://doi.org/10.1371/journal.pone.0199658.g003

thumbnail
Fig 4. Variables distribution.

Distribution of the variables in the final dataset.

https://doi.org/10.1371/journal.pone.0199658.g004

Gender differences

We conducted t-tests to compare the scores of male and female participants. As shown in Table 5A, we did not find any significant differences suggesting that male and female participants rated pain words similarly. That ratings of male and female participants did not differ is also confirmed by the significantly high positive correlations of the ratings of female and male participants for all the variables [Familiarity (r = .884, p < .001), AoA (r = .963, p < .001), Imageability (r = .906, p < .001), Concreteness (r = .917, p < .001), Context Availability (r = .826, p < .001), Valence (r = .941, p < .001), Arousal (r = .846, p < .001), Pain-relatedness (r = .904, p < .001), Intensity (r = .899, p < .001), Unpleasantness (r = .918, p < .001)].

thumbnail
Table 5. Descriptive statistics and t-test concerning the ratings provided by male and female responders.

https://doi.org/10.1371/journal.pone.0199658.t005

It should be noted that also the original ANEW study [85] did not report any significant gender difference. In the Italian adaptation of the ANEW instead, Montefinese et al. [7] did find a significant gender difference on Arousal ratings, although the ratings were highly correlated (note that we did not test Dominance for which Montefinese et al. also reported a significant gender difference).

To further investigate potential gender differences, we also analyzed separately the ratings provided by female and male responders to physical and social pain words (Table 5B and 5C, respectively). Three significant differences emerged, all concerning social pain words. Female participants provided higher ratings of Arousal than male participants (see also [7]). In addition, female participants rated social pain words as more associated to pain and conveying more intense pain than male responders.

Table 5 refers to all the words together, as well as to physical pain words and social pain words alone.

Hierarchical clustering analysis

We also conducted a Hierarchical Clustering Analysis (HCA; Fig 5; [127]) that is ideal for exploring the correlational structure of the 16 measures used in this study. Hierarchical Clustering Analysis (HCA) is the general name of a family of techniques aimed at unveiling the underlying structure of a multivariate dataset by displaying it in a tree-like format [127]. HCA has the advantage of bringing out the main clusters in the data more clearly [128] and is particularly well suited to explore the correlational structure of a large number of measures. The dendrogram resulting from the HCA (Fig 5) shows that the highest split separates the lexical variables, the sub-lexical variables, Familiarity, AoA, and Context Availability on the one hand, from affective and pain-related variables, Imageability and Concreteness on the other hand. Within the former branch, Familiarity, AoA, and Context Availability cluster together, presumably because familiar words often are also acquired earlier and easier to contextualize. Word frequency (Zipf) stands on the top of this cluster. Another cluster is formed by distributional variables such as Neighbor Mean Frequency, Word Length, Neighbor Max Frequency, NSize, and OLD20. Interestingly, NSize and OLD20 are recognized as different metrics for the same construct (which they are indeed; e.g., [120]). It is not entirely clear what psychological construct this cluster may tap on. One possibility is that the core of the cluster is represented by Word Length, which strongly determines the features of a word’s lexical neighborhood. Within the second main branch, there is a cluster containing Imageability and Concreteness ratings, which is separated from the cluster relative to affective and pain-related variables. Interestingly, the structure of the affective and pain-related branch of the tree suggests that Pain-relatedness and Intensity are hardly separable. Differently, Unpleasantness stands alone, emerging as a distinct variable, albeit strongly correlated with the other two pain-related variables. That Intensity and Unpleasantness stand separately is consistent with experimental studies on pain showing that these two variables can be dissociated since they reflect two distinct components of pain (the sensory-discriminative component and the affective-motivational component, respectively) [129,130].

thumbnail
Fig 5. Hierarchical Clustering Analysis dendrogram.

Dendrogram resulting from the Hierarchical Clustering Analysis of the 16 variables.

https://doi.org/10.1371/journal.pone.0199658.g005

Partial correlation analyses

In what follows, we describe the results of the partial correlations among the variables (Fig 6 and Table 6). To avoid the problem of multicollinearity among Pain-relatedness, Intensity and Unpleasantness (r > .9), in these analyses we only used Pain-relatedness ratings. Moreover, given the high number of comparisons carried out (i.e., 91), we used a Bonferroni-corrected α value of .05/91 ≈ .0006. Finally, we present the results of separate one-way ANOVAs on the mean ratings of each variable for nouns, adjectives, and verbs and then for physical and social pain words.

thumbnail
Fig 6. Partial correlations among all the variables.

The dot color indicates the direction of the correlation (blue for direct, red for inverse) and the size and transparency its strength.

https://doi.org/10.1371/journal.pone.0199658.g006

Partial correlations among psycholinguistic variables.

Partial correlation analyses (Table 6) revealed that more familiar words are learnt earlier in life (r = -.38) and are more prone to elicit a context (r = .64). In fact, Familiarity inversely correlates with AoA and positively correlates with Context Availability [31,131133]. The more familiar pain words are, the less imaginable and concrete are (r = -.22 and r = -.17, respectively). Admittedly, we do not have an explanation for the significant inverse correlations between Familiarity and Imageability, and between Familiarity and Concreteness, which are inconsistent with what is typically reported in the literature on affective words (e.g., [31,67,132,133]; but see [131]) (we return on this point in the Conclusions). Further analyses conducted on the three word classes and on physical and social pain separately are shown in S1 Table and revealed that these two inverse correlations are statistically significant only for nouns (and not for adjectives and verbs) and specifically only for physical pain nouns. One possibility is that these inverse correlations reflect the specific type of affective nouns tested in this study. In fact, the words that we most often use to convey physical pain include a variety of nouns as, for instance, names of syndrome and illness (e.g., gastrite, gastritis) and generic terms (e.g., acciacco, infirmity) that are hardly concrete and imageable.

Frequency is significantly correlated only with Familiarity [31,131,133] in that the more frequent a word is, the more familiar it is rated (r = .23), quite unsurprisingly. Words learnt earlier in life are also rated as more imaginable (r = -.29), in line with the literature [31,108,131,133]. Again in line with the literature [113,131,134,135], the more a pain word is concrete, the more it is imageable and prone to elicit a context (r = .28 and r = .19, respectively). Positive correlations between Imageability and Concreteness for affective words have been reported in a variety of languages, including English [104,135], Chinese [136], European Portuguese [89], French [137], and Spanish [138]. Finally, longer words are rated as more familiar and with smaller neighborhoods and higher OLD20 values.

Partial correlations between affective and pain-related variables.

According to the literature on affective words [7,32,70,71,99,139,140], valence and arousal ratings typically exhibit a U-shaped relationship whereby highly valenced words (both positive and negative) also have higher arousal ratings than neutral words. The bivariate correlation between Valence and Arousal ratings of pain-related words reveals a significant linear rather than a quadratic relationship (r = -.56). The bivariate correlation between Valence and Arousal ratings of pain-related words (Fig 7) reveals a significant linear rather than a quadratic relationship (r = -.56), possibly representing the negative portion of the classic U-shaped relationship.

thumbnail
Fig 7. Partial scatterplot.

Partial scatterplot of mean values in the Valence and Arousal dimensions, along with the quadratic regression line (R2 = .33).

https://doi.org/10.1371/journal.pone.0199658.g007

This database is about pain words which of course moves the valence distribution towards its negative end. However, this correlation is not significant anymore after controlling for the effects of psycholinguistic and pain-related variables. Partial correlations, instead, reveal that the more a word is associated to pain, the more negative and arousing it is. In fact, Pain-relatedness inversely correlates with Valence and positively correlates with Arousal. This is consistent with studies on emotionally-laden words showing that an increase in negative valence is often associated to an increase in arousal (e.g., [7,8,69,71,88]).

Partial correlations among psycholinguistic, affective, and pain-related variables.

The more positive a word is, the more concrete it is rated, as shown by a positive correlation between Valence and Concreteness. This result is consistent with prior studies showing a joint effect of valence and concreteness on word recognition in a variety of tasks (for an overview, see [141]). Finally, the more a word is associated to pain, the more it is rated as concrete and acquired later in life, as shown by positive correlations between Pain-relatedness and AoA, and Pain-relatedness and Concreteness.

Differences among word classes

Our database is composed by 42.6% of adjectives, 38.9% of nouns, 14.6% of verbs and 3.9% of ambiguous words (i.e., adjectives that can also be used as nouns). Since grammatical class is known to affect linguistic processing, and specifically that of affective words [98], we conducted separate by-item one-way ANOVAs on each variable with Word Class (Adjectives vs. Nouns vs. Verbs) as a between-item factor.

The one-way ANOVA on AoA reveals a statistically significant difference among word classes [F (2,488) = 9.564, p < .001, η2 = .038]. Post-hoc comparisons (with the Tukey HSD test) show that verbs (M = 4.13, SD = 1.46) are learnt significantly earlier than both nouns (M = 4.65, SD = 1.38, p = .015) and adjectives (M = 4.92, SD = 1.3, p < .001). This is likely to reflect the specific semantic domain tested in this study. In fact, while many of the nouns referring to pain concern events or experiences predominantly occurring in adulthood (e.g., tremore, tremor; abbandono, neglect), verbs describe actions that are rather common in the childhood (e.g., scivolare, to slip; cadere, to fall; graffiare, to scratch).

One-way ANOVAs show significant word class effects also on Imageability [F(2,489) = 106.105, p < .001, η2 = .303], Concreteness [F (2,489) = 138.229, p < .001, η2 = .361], and Context Availability [F(2,489) = 54.733, p < .001, η2 = .183]. In fact, adjectives are rated as significantly less imaginable, concrete and also less prone to elicit a context than nouns and verbs.

One-way ANOVAs on Valence [F(2,487) = 39.592, p < .001, η2 = .14] and Arousal [F(2,489) = 29.274, p < .001, η2 = .11] show significant word class effects as well. Adjectives are rated as more positive and less arousing than nouns and verbs. This may reflect the fact that a consistent number of our adjectives can be used to modify pain-unrelated nouns as well (e.g., grande, big, acuto, acute). In fact, 78 of the 218 adjectives are rated as weakly or not at all associated to pain (Pain-relatedness < 3). Moreover, verbs are rated as significantly more arousing than nouns (p < .001), reflecting the action-oriented nature of most of our verbs.

One-way ANOVAs on Pain-relatedness [F(2,489 = 57.79, p < .001, η2 = .191], Intensity [F (2,489) = 44.354, p < .001, η2 = .154] and Unpleasantness [F(2,489) = 36.806, p < .001, η2 = .131] again reveal significant effects of word class. Unsurprisingly, adjectives are judged as significantly less pain-related and conveying a less intense and unpleasant pain than nouns and verbs. Again, this may reflect the fact that many of our adjectives have a general semantic scope (e.g., grande, big; immenso, immense). ANOVA on Familiarity does not reveal any significant differences among the three word classes [F(2,489) = 1.114, p = .329, η2 = .005].

Partial correlations for nouns, adjectives, and verbs are reported in S1 Table.

Differences between words conveying physical and social pain

In order to understand whether the psycholinguistic and affective properties of physical and social pain words differ, we conducted by-item one-way ANOVAs on each variable with Type of Pain (Physical vs. Social) as a between-item factor.

One-way ANOVAs on Concreteness [F(1,510) = 21.112, p < .001, η2 = .04], Valence [F(1,508) = 52.77, p < .001, η2 = .094], Pain-relatedness [F(1,510) = 6.352, p = .012, η2 = .012], Intensity [F(1,510) = 10.136, p = .002, η2 = .019], and Unpleasantness [F(1,510) = 28.377, p < .001, η2 = .053] yield statistically significant differences. Specifically, the words conveying social pain are rated as less concrete, but more negative, than the words conveying physical pain. Interestingly, participants rate social pain words as more associated to pain, and conveying a more intense and unpleasant pain, than physical pain words.

ANOVAs on Familiarity [F(1,510) = .001, p = .97, η2 = .000], AoA [F(1,510) = 2.720, p = .397, η2 = .001], Imageability [F(1,510) = 3.498, p = .062, η2 = .007], Context Availability [F(1,510) = .436, p = .509, η2 = .001], and Arousal [F(1,510) = 2.104, p = .148, η2 = .004] do not reveal any significant differences between physical and social pain.

Conclusions

The aim of the present study was twofold. First, we assessed the psycholinguistic, affective, and pain-related characteristics of Italian words conveying physical and social pain providing a normed lexicon of pain. Second, we explored the relationships among these variables unveiling important aspects of the lexico-semantic architecture underlying the Italian pain lexicon. To these aims, we collected ratings for psycholinguistic, affective and pain-related variables, as well as distributional data, for 512 words expressing physical and social pain. These norms respond to the need for normed stimuli to be used in the experimental research on pain and on negative affect in Italian.

We carried out a Hierarchical Clustering Analysis (HCA) to explore the structure underlying the correlations among the 16 variables measured in this study. Two interesting results emerge from the HCA. The first is that pain-related variables cluster separately from all the other variables. The second interesting result concerns the organization of pain-related variables that shows two different clusters: Unpleasantness, that clusters by itself pointing to the affective-motivational dimension of pain, and Intensity and Pain-relatedness that cluster together pointing to the sensory-discriminative dimension of pain.

In line with prior studies on the affective lexicon, we found that the pain words acquired earlier in life are also more familiar and imageable [31,67,131,133,142], and that more familiar words are also more easily associated to specific contexts. More imaginable words are also rated as more concrete [113,131,134,135] and more prone to elicit a context. At variance with the literature [108,131], we found that the more physical pain nouns are familiar, the less imaginable and concrete they are rated. Admittedly, we do not yet have an explanation for these results. One possibility is that they may reflect the semantic heterogeneity of the nouns of this corpus that include medical terms (e.g., gastrite, gastritis), illness generic nouns and lay person pain words (e.g., acciacco, infirmity) not easily classifiable as imageable and/or concrete. In addition, responders may know the names of painful events, states or illnesses they have never directly experienced hence diminishing their ability to decide how much they are concrete and to image them. Even the words pain or disease refer to generic, intangible, and poorly delineated experiences, not directly observable [11], that are likely to be considered scarcely concrete and/or imageable.

Verbs conveying actions that may cause pain, or represent antecedents of pain experiences, are judged to have been acquired earlier than adjectives and nouns. This suggests that the development of a more sophisticated pain-related lexicon emerges as we grow up. This lexicon is used to convey a broad range of painful experiences, including those producing social pain. This is confirmed by the positive correlation between Pain-relatedness and AoA that reveals that the words more associated to pain are also judged to be learnt later in life.

Social pain words are rated as more negative and pain-related than physical pain words, and as reflecting more intense and unpleasant pain experiences than physical pain words. This is likely to reflect the relatively young age of our responders for whom social pain could represent a more salient and frequent experience than physical pain. In fact, 17.8% of the responders answered that they currently suffer of chronic pain and 5.1% of chronic pain in the past. These percentages are important but in any case lower than the mean incidence of chronic pain in the Italian population that concerns the 26% of Italians [143]. However, since the question was phased rather generically without specifically listing what could count as “chronic pain”, or the types of experienced chronic pain, we cannot be sure that indeed it was selected by responders suffering chronic pain as defined in the clinical literature. In any case, since a qualitative inspection of the results of the two subsets of participant (i.e., responders with and without actual/past chronic pain) did not suggest any differences in the distribution of the ratings of the variable tested, they were analyzed all together. However, a possible important effect of age on physical vs. social pain perception may not represent the whole story. In fact, a wealth of studies about the subjective impact of social pain has documented that often this is considered as much threatening and important as physical pain. Notably, nearly three out of four people listed the loss of a close relationship for death or relationship break-up as the “single most negative emotional event” of their lives [56,144]. A study administering the same questionnaires to older participants (41–70 years) is currently in progress to clarify whether the higher negativity and Pain-relatedness of social pain words indeed depend on the age of responders.

One might wonder whether suffering or having suffered of chronic pain may have a general effect on the ratings provided for physical pain words. Assessing whether participants in the study have, or have had, painful experiences, either physical or social, would be crucial to clarify this point. However, as we mentioned, we only asked generically if the responders suffered or had ever suffered of chronic pain and we did not investigate at all whether responders suffered of had ever suffered of social pain. Admittedly, this is an important limitation of this study. In fact, the possibility exists that both forms of pain may affect the ways in which we linguistically categorize and evaluate pain. We are currently running a study on cancer patients where we administer them an adapted form of the WOP. This could clarify whether a condition of severe oncological pain affects the semantic of pain. We expect that this may be the case since pain is intimately associated with alterations of physiological and psychological processes of pain perceptions and pain-related behaviors [145,146].

The biological gender of participants does not seem to affect the results of our study, differently from what was found for Italian affectively-laden words by Montefinese et al. [7], although only for arousal. However, as Montefinese et al. clarified, these gender differences are moderated by the high correlation between male and female ratings of arousal found in the study. A growing body of research about the role of gender differences in medical language and communication has reported gender differences in the affective and social content of symptoms descriptions, willingness to report pain, and words used to describe pain [147,148]. These differences have been linked to psycho-social gender roles. However, these gender differences may not necessarily lead to different ratings of the psycholinguistic and affective variables tested in this study [149]. In addition, we cannot exclude the possibility that gender differences in pain communication could emerge once pain and illness have been consistently experienced, usually later in life. However, due to the online recruitment of responders that reflected the preponderance of female students, we did not have the same number of male and female participants. Although we cannot exclude that this may have influenced the lack of significant gender differences, it should be noted that other more gender-balanced studies on the affective lexicon did not find gender differences either.

Pain words belong to the realm of negative words. Interestingly, our results suggest that not all pain words seem to be negative alike. For instance, the words associated to labor pain (e.g., partorire, to give birth; doglia, labor pain) are rated as extremely intense and unpleasant but with a predominantly positive Valence. Interestingly, these word ratings are similar to the ratings of Intensity, Unpleasantness and Valence reported in the literature on labor pain. In fact, when asked to evaluate their childbirth experience, women rated it as extremely high in Intensity, but lower in Unpleasantness than other types of pain, and having a positive Valence [150].

Consensus exists that stimuli are automatically evaluated in terms of their affective valence [151,152] along a negative-to-positive valence gradient [22,116,153,154]. So far, studies on valenced words have predominantly treated negative words as a unitary category. However, recently it has been suggested that negative words may not represent a unitary category but rather they may differ based on their specific semantic content [155,156]. For instance, a recent brain-imaging meta-analysis has shown that the brain did not treat negative stimuli (be they words or images) as a unified class [157159]. One can speculate that pain words may represent a domain with a specific status among negatively valenced words due to the high relevance of pain experiences in everyday life and for survival. Future studies devoted to test this aspect are required before one can draw any firm conclusions.

WOP provides norms about the specific part of our lexicon in that convey physical and social pain. We obviously see this as an important strength of this work. However, this also determined the presence of a few positively valenced words in our database. This limitation is mitigated by the fact that 78 of the 218 adjectives of WOP can be used to modify pain-related as well as pain-unrelated nouns (e.g., immenso, immense, grande, big, infinito, infinite). In fact, 51 out of these 78 adjectives were rated as positive together with the noun parto (delivery) and the verb partorire (to deliver). Nine adjectives were rated as neutral, together with the verbs grattare (to scratch) and stringere (to tighten). The general Valence distribution of our stimuli is indeed a little skewed towards the negative end (mean = -.9, median = -1.3), but covers the entire range of possible values (min = -2.97, max = +2.52). A similar consideration applies to Pain-relatedness that may be expected to peak very narrowly around high values; but it did not. In fact, Pain-relatedness ranged from 1.16 to 6.83, with a mean value of 4.34 and median value of 4.43, mostly thanks to adjectives. Therefore, although the database is obviously tight to the specific investigation of pain words, it does provide a wider spectrum of stimuli.

Finally, we acknowledge that we had a different number of observations per cell for some stimuli and that this may represent a problem. However, our ratings were provided by at least 31 responders which represents a reasonable number of observations compared to other databases (for instance, the Italian version of ANEW provides affective ratings from at least 31 participants and psycholinguistics ratings from 20 participants).

To the best of our knowledge, this is the first descriptive study on the psycholinguistic, affective, and pain-related characteristics of physical and social pain words. This normative study provides a useful tool that may enable researchers to use highly controlled stimuli in experimental studies on physical and social pain as well as on language and negative affect.

Supporting information

S1 Text. Original, Italian survey instruction are provided together with the English translation.

https://doi.org/10.1371/journal.pone.0199658.s001

(DOCX)

S1 Fig. Example of the Rodriguez and Laio clustering procedure.

An example of the Rodriguez and Laio clustering procedure using Familiarity ratings (on a 7-point scale) for the words “correlation” and “variance” given by ten participants (from subject 1 to subject 10). They are represented as points in a two–dimensional space, and their position is defined by their ratings. Subjects 1 to 4 (s1–s4, in green color) gave consistent, high judgments; subjects 5 to 8 (s5–s8, in blue color) also gave consistent, low judgments. Conversely, subjects 9 and 10 (s9–s10, in red color) provided highly idiosyncratic responses, as indicated by their isolated position on the graph.

https://doi.org/10.1371/journal.pone.0199658.s002

(DOCX)

S1 Table. Partial correlations among all the variables of interest considering the three word classes and physical and social pain separately.

Table 1A refers to partial correlations for nouns. Table 1B refers to partial correlations for adjectives. Table 1C refers to partial correlations for verbs. Table 1D refers to partial correlations for physical pain words. Table 1E refers to partial correlations for social pain word. Table 1F refers to partial correlations for physical pain nouns. Abbreviations: Subtlex-IT Frequency (Zipf), Neighborhood Size (N), Orthographic Levenshtein Distance 20 (OLD20), Neighbor Max Frequency (MaxFreqN), Neighbor Mean Frequency (MeanFreqN).

https://doi.org/10.1371/journal.pone.0199658.s003

(DOCX)

References

  1. 1. Richter M, Eck J, Straube T, Miltner WH, Weiss T. Do words hurt? Brain activation during the processing of pain-related words. Pain. 2010 Feb;148(2):198–205. pmid:19846255
  2. 2. Richter M, Schroeter C, Puensch T, Straube T, Hecht H, Ritter A, et al. Pain-related and negative semantic priming enhances perceived pain intensity. Pain Res Manag. 2014 Mar-Apr;19(2):69–74. pmid:24716197
  3. 3. Ritter A, Franz M, Puta C, Dietrich C, Miltner WH, Weiss T. Enhanced brain responses to pain-related words in chronic back pain patients and their modulation by current pain. Healthcare (Basel). 2016 Aug 10;4(3).
  4. 4. Melzack R. Pain measurement and assessment. New York: Raven Press; 1983.
  5. 5. Macdonald G, Leary MR. Why does social exclusion hurt? The relationship between social and physical pain. Psychol Bull. 2005 Mar;131(2):202–23. pmid:15740417
  6. 6. Wang W, Kennedy R, Lazer D, Ramakrishnan N. Growing pains for global monitoring of societal events. Science. 2016 Sep 30;353(6307):1502–3. pmid:27708093
  7. 7. Montefinese M, Ambrosini E, Fairfield B, Mammarella N. The adaptation of the affective norms for English words (ANEW) for Italian. Behav Res Methods. 2014 Sep;46(3):887–903. pmid:24150921
  8. 8. Fairfield B, Ambrosini E, Mammarella N, Montefinese M. Affective norms for Italian words in older adults: age differences in ratings of valence, arousal and dominance. PLoS One. 2017 Jan 3;12(1):e0169472. pmid:28046070
  9. 9. Gu X, Han S. Neural substrates underlying evaluation of pain in actions depicted in words. Behav Brain Res. 2007 Aug 6;181:218–23. pmid:17512615
  10. 10. Osaka N, Osaka M, Morishita M, Kondo H, Fukuyama H. A word expressing affective pain activates the anterior cingulate cortex in the human brain: an fMRI study. Behav Brain Res. 2004 Aug 12;153:123–127. pmid:15219713
  11. 11. Semino E. Descriptions of pain, metaphor and embodied simulation. Metaphor Symb. 2010;25(4):205–26.
  12. 12. Avenanti A, Minio-Paluello I, Bufalari I, Aglioti SM. Stimulus-driven modulation of motor-evoked potentials during observation of others’ pain. Neuroimage. 2006 Aug 1;32:316–324. pmid:16675270
  13. 13. Betti V, Aglioti SM. Dynamic construction of the neural networks underpinning empathy for pain. Neurosci Biobehav Rev. 2016 Apr;63:191–206. pmid:26877105
  14. 14. Bradshaw JL, Mattingley JB. Allodynia: a sensory analogue of motor mirror neurons in a hyperaesthetic patient reporting instantaneous discomfort to another’s perceived sudden minor injury? J Neurol Neurosurg Psychiatry. 2001 Jun 1;70:135–6.
  15. 15. Fitzgibbon BM, Giummarra MJ, Georgiu-Karistianis N, Enticott PG, Bradshaw JL. Shared pain: from empathy to synaesthesia. Neurosci Biobehav Rev. 2010 March;34:500–12. pmid:19857517
  16. 16. Osborn J, Derbyshire SWG. Pain sensation evoked by observing injury in others. Pain. 2010 Feb;148(2):268–274. pmid:20005042
  17. 17. Stanford EA, Chambers CT, Craig KD. A normative analysis of the development of pain-related vocabulary in children. Pain. 2005 Mar;144(1–2):278–84.
  18. 18. Gracely RH. Pain language and evaluation. Pain. 2016 Jul 7;157(7),1369–72.
  19. 19. Main CJ. Pain assessment in context: a state of the science review of the McGill Pain Questionnaire 40 years on. Pain. 2016 Jul;157(7):1387–99. pmid:26713423
  20. 20. Melzack R. The McGill Pain Questionnaire: major properties and scoring methods. Pain. 1975 Sep;1(3):277–99. pmid:1235985
  21. 21. Feldman-Barrett L, Russell JA. Independence and bipolarity in the structure of current affect. J Pers Soc Psychol. 1988 April;74(4),967–84.
  22. 22. Russell JA. A circumplex model of affect. J Pers Soc Psychol. 1980;39,1161–78.
  23. 23. Osgood CE, Suci GJ, Tannenbaum PH. The measurement of meaning. Oxford, England: Univer. Illinois Press; 1967.
  24. 24. Mather M, Sutherland MR. Arousal-biased competition in perception and memory. Perspect Psychol Sci. 2011 Mar;6(2):114–33. pmid:21660127
  25. 25. Reisenzein R. Pleasure-Arousal theory and the intensity of emotions. J Pers Soc Psychol. 1994;67(3):525–39.
  26. 26. Russell JA. Core affect and the psychological construction of emotion. Psychol Rev. 2003 Jan;110(1):145–72. pmid:12529060
  27. 27. Recio G, Conrad M, Hansen LB, Jacobs AM. On pleasure and thrill: the interplay between arousal and valence during visual word recognition. Brain Lang. 2014 Jul;134:34–43. pmid:24815948
  28. 28. Citron FMM. Neural correlates of written emotion word processing: A review of recent electrophysiological and hemodynamic neuroimaging studies. Brain Lang. 2012 Sep;122(3):211–26. pmid:22277309
  29. 29. Vinson D, Ponari M, Vigliocco G. How does emotional content affect lexical processing? Cogn Emot. 2014;28(4):737–46. pmid:24215294
  30. 30. Abbassi E, Kahlaoui K, Wilson MA, Joanette Y. Processing the emotions in words: the complementary contributions of the left and right hemispheres. Cogn Affect Behav Neurosci. 2011 Sep;11(3):372–85. pmid:21533883
  31. 31. Citron FMM, Weekes BS, Ferstl EC. How are affective word ratings related to lexicosemantic properties? Evidence from the Sussex Affective Word List (SAWL). Appl Psycholinguist. 2014 March;35(2):313–31.
  32. 32. Herbert C, Junghofer M, Kissler J. Event-related potentials to emotional adjectives during reading. Psychophysiology. 2008 May;45(3):487–98. pmid:18221445
  33. 33. Vigliocco G, Kousta ST, Della Rosa PA, Vinson DP, Tettamanti M, Devlin JT, et al. The neural representation of abstract words: the role of emotion. Cereb Cortex. 2014 Jul;24(7):1767–77. pmid:23408565
  34. 34. Lai S, Liu K, Xu L, Zhao J. How to generate a good word embedding? Intelligent Systems, IEEE. July 2015; 31(6).
  35. 35. Dahl M. Asymmetries in the processing of emotionally valenced words. Scand J Psychol. 2001 Apr;42(2):97–104. pmid:11321641
  36. 36. Ito TA, Larsen JT, Smith NK, Cacioppo JT. Negative information weighs more heavily on the brain: the negativity bias in evaluative categorizations. J Pers Soc Psychol. 1998 Oct;75(4):887–900. pmid:9825526
  37. 37. Öhman A, Mineka S. Fears, phobias, and preparedness: Toward an evolved module of fear and fear learning. Psychol Rev. 2001 Jul;108(3):483–522. pmid:11488376
  38. 38. Pratto F, John OP. Automatic vigilance: the attention-grabbing power of negative social information. J Pers Soc Psychol. 1991 Sep;61(3):380–91. pmid:1941510
  39. 39. Wentura D, Rothermund K, Bak P. Automatic vigilance: the attention-grabbing power of approach-and avoidance-related social information. J Pers Soc Psychol. 2000 Jun;78(6):1024–37. pmid:10870906
  40. 40. Fox E, Russo R, Bowles R, Dutton K. Do threatening stimuli draw or hold visual attention in subclinical anxiety? J Exp Psychol Gen. 2001 Dec;130(4):681–700. pmid:11757875
  41. 41. Mckenna FP, Sharma D. Reversing the emotional Stroop effect reveals that it is not what it seems: the role of fast and slow components. J Exp Psychol Learn Mem Cogn. 2004 Mar;30(2):382–92. pmid:14979812
  42. 42. Most SB, Scholl BJ, Clifford ER, Simons DJ. What you see is what you set: Sustained inattentional blindness and the capture of awareness. Psychol Rev. 2005 Jan;112(1):217–42. pmid:15631594
  43. 43. Williams JM, Mathews A, MacLeod C. The emotional Stroop task and psychopathology. Psychol Bull. 1996 Jul;120(1):3–24. pmid:8711015
  44. 44. Larsen RJ, Mercer KA, Balota DA. Lexical characteristics of words used in emotional Stroop experiments. Emotion. 2006 Feb;6(1):62–72 pmid:16637750
  45. 45. Hofmann M. J., Kuchinke L., Tamm S., Võ M. L.-H., Jacobs A. M. (2009). Affective processing within 1/10th of a second: High arousal is necessary for early facilitative processing of negative but not positive words. Cogn Affect Behav Neurosci. 2009 Dec;9(4):389–97. pmid:19897792
  46. 46. Algom D, Chajut E, Lev S. A rational look at the emotional Stroop phenomenon: a generic slowdown, not a Stroop effect. J Exp Psychol Gen. 2004 Sep;133(3):323–38. pmid:15355142
  47. 47. Knickerbocker H, Johnson RL, Altarriba J. Emotion effects during reading: influence of an emotion target word on eye movements and processing. Cogn Emot. 2015;29(5):784–806. pmid:25034443
  48. 48. Bradley MM. Emotion and motivation. In: Cacioppo JT, Tassinary LG, Berntson G, editors. Handbook of Psychophysiology New York, NY: Cambridge University Press; 2000. pp. 581–607.
  49. 49. Estes Z, Adelman JS. Automatic vigilance for negative words is categorical and general. Emotion. 2008;8(4):453–457.
  50. 50. Kuperman V, Estes Z, Brysbaert M, Warriner AB. Emotion and language: valence and arousal affect word recognition. J Exp Psychol Gen. 2014 Jun;143(3):1065–1081. pmid:24490848
  51. 51. Nasrallah M, Carmel D, Lavie N. Murder, she wrote: enhanced sensitivity to negative word valence. Emotion. 2009 Oct;9(5):609–18. pmid:19803583
  52. 52. Estes Z, Adelman JS. Automatic vigilance for negative words in lexical decision and naming: comment on Larsen, Mercer, and Balota (2006). Emotion. 2008 Aug;8(4):441–4; discussion 445–57. pmid:18729575
  53. 53. Kousta ST, Vinson DP, Vigliocco G. Emotion words, regardless of polarity, have a processing advantage over neutral words. Cognition. 2009 Sep;112(3):473–81. pmid:19591976
  54. 54. Eisenberger NI, Lieberman MD. Why rejection hurts: a common neural alarm system for physical and social pain. Trends Cogn Sci. 2004 Jul;8(7):294–300. pmid:15242688
  55. 55. Eisenberger NI. The neural bases of social pain: evidence for shared representations with physical pain. Psychosom Med. 2012 Feb-Mar;74(2):126–35. pmid:22286852
  56. 56. Jaremka LM, Gabriel S, Carvallo M. What makes us feel the best also makes us feel the worst: the emotional impact of independent and interdependent experiences. Self and Identity. 2011;10(1):44–63.
  57. 57. Cristofori I, Moretti L, Harquel S, Posada A, Deiana G, Isnard J, et al. Theta signal as the neural signature of social exclusion. Cereb Cortex. 2013 Oct;23(10):2437–47. pmid:22875860
  58. 58. Wierzbicka A. Is pain a human universal? A cross-linguistic and cross-cultural perspective on pain. Emotion Review. 2012;4(3):307–317.
  59. 59. Clinton H. Clinton’s emotional concession: This is “painful and will be for a long time” (Concession speech after 2016 US election). 2006 Nov 09. https://www.usatoday.com/story/news/politics/elections/2016/11/09/hillary-clinton-concession-speech/93536168/
  60. 60. Eisenberger NI. Social pain and the brain: controversies, questions, and where to go from here. Annu Rev Psychol. 2015 Jan 3;66:601–29. pmid:25251482
  61. 61. Cacioppo S, Frum C, Asp E, Weiss RM, Lewis JW, Cacioppo JT. A quantitative meta-analysis of functional imaging studies of social rejection. Sci Rep. 2013;3:2027. pmid:24002359
  62. 62. Balota DA, Cortese MJ, Sergent-Marshall SD, Spieler DH, Yap MJ. Visual word recognition of single-syllable words. J Exp Psychol Gen. 2004 Jun;133(2):283–316. pmid:15149254
  63. 63. Balota DA, Yap MJ, Hutchison KA, Cortese MJ, Kessler B, Loftis B, et al. The English Lexicon Project. Behav Res Methods. 2007 Aug;39(3):445–59. pmid:17958156
  64. 64. Ferrand L, New B, Brysbaert M, Keuleers E, Bonin P, Méot A, et al. The French Lexicon Project: lexical decision data for 38,840 French words and 38,840 pseudowords. Behav Res Methods. 2010 May;42(2):488–96. pmid:20479180
  65. 65. Keuleers E, Diependaele K, Brysbaert M. Practice effects in large-scale visual word recognition studies: a lexical decision study on 14,000 Dutch mono-and disyllabic words and nonwords. Front Psychol. 2010 Nov 18;1:174. pmid:21833236
  66. 66. Brysbaert M, Warriner AB, Kuperman V. Concreteness ratings for 40 thousand generally known English word lemmas. Behav Res Methods. 2014 Sep;46(3):904–11. pmid:24142837
  67. 67. Kuperman V, Stadthagen-Gonzalez H, Brysbaert M. Age-of-acquisition ratings for 30,000 English words. Behav Res Methods. 2012 Dec;44(4):978–90. pmid:22581493
  68. 68. Lahl O, Göritz AS, Pietrowsky R, Rosenberg J. Using the World-Wide Web to obtain large-scale word norms: 190,212 ratings on a set of 2,654 German nouns. Behav Res Methods. 2009 Feb;41(1):13–19. pmid:19182119
  69. 69. Warriner AB, Kuperman V, Brysbaert M. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behav Res Methods. 2013 Dec;45(4):1191–207. pmid:23404613
  70. 70. Võ MLH, Conrad M, Kuchinke L, Urton K, Hofmann MJ, Jacobs AM. The Berlin affective word list reloaded (Bawl-R). Behav Res Methods. 2009 May;41(2):534–8. pmid:19363195
  71. 71. Võ MLH, Jacobs AM, Conrad M. Cross-validating the Berlin Affective Word List. Behav Res Methods. 2006 Nov;38(4):606–9. pmid:17393831
  72. 72. Briesemeister BB, Kuchinke L, Jacobs AM. Discrete emotion norms for nouns: Berlin affective word list (DENN–BAWL). Behav Res Methods. 2011 Jun;43(2):441–8. pmid:21416309
  73. 73. Cortese MJ, Fugett A. Imageability ratings for 3,000 monosyllabic words. Behav Res Methods Instrum Comput. 2004 Aug;36(3):384–7. pmid:15641427
  74. 74. Citron FMM, Cacciari C, Kucharski M, Beck L, Conrad M, Jacobs AM. When emotions are expressed figuratively: Psycholinguistic and affective norms of 619 idioms for German (PANIG). Behav Res Methods. 2016 Mar;48(1):91–111. pmid:25821142
  75. 75. Ferré P, Guasch M, Moldovan C, Sánchez-Casas R. Affective norms for 380 Spanish words belonging to three different semantic categories. Behav Res Methods. 2012 Jun;44(2):395–403. pmid:22042646
  76. 76. Montefinese M, Ambrosini E, Fairfield B, Mammarella N. Semantic memory: A feature-based analysis and new norms for Italian. Behav Res Methods. 2013 Jun;45(2):440–6 pmid:23055168
  77. 77. Barca L, Burani C, Arduino LS. Word naming times and psycholinguistic norms for Italian nouns. Behav Res Methods Instrum Comput. 2002 Aug;34(3):424–34. pmid:12395559
  78. 78. Baroni M, Bernardini S, Comastri F, Piccioni L, Volpi A, Aston G, et al. Introducing the La Repubblica Corpus: a large, annotated, TEI(XML)-compliant corpus of newspaper Italian. LREC.
  79. 79. Bertinetto PM, Burani C, Laudanna A, Marconi L, Ratti D, Rolando C, et al. Corpus e Lessico di Frequenza dell’Italiano Scritto (CoLFIS). Retrieved from: http://linguistica.sns.it/CoLFIS/Home.htm
  80. 80. Crepaldi D, Amenta S, Pawel M, Keuleers E, Brysbaert M. SUBTLEX-IT. Subtitle-based word frequency estimates for Italian. Proceedings of the Annual Meeting of the Italian Association For Experimental Psychology; 2015 Sep 10–12; Rovereto (Italy).
  81. 81. Della Rosa PA, Catricalà E, Vigliocco G, Cappa SF. Beyond the abstract-concrete dichotomy: mode of acquisition, concreteness, imageability, familiarity, age of acquisition, context availability, and abstractness norms for a set of 417 Italian words. Behav Res Methods. 2010 Nov;42(4):1042–8. pmid:21139171
  82. 82. Guida A, Lenci A. Semantic properties of word associations to Italian verbs. Italian Journal of Linguistics. 2007;19(2):293–326.
  83. 83. Lebani GE, Bondielli A, Lenci A. You are what you do: An empirical characterization of the semantic content of the thematic roles for a group of Italian verbs. J Cogn Sci. 2015;16(4):399–428.
  84. 84. Marelli M. Word-Embeddings Italian Semantic Spaces: a semantic model for psycholinguistic research. Psihologija. 2017;50(4);503–20
  85. 85. Bradley MM, Lang PPJ. Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings. Psychology. 1999; Technical: 0.
  86. 86. Kristensen CH, Falcão de Azevedo Gomes C, Reuwsaat Justo A, Vieira K. Brazilian norms for the Affective Norms for English Words. Trends Psychiatry Psychother. 2011;33(3):135–46. pmid:25924086
  87. 87. Wei WL, Wu CH, Lin JC. A regression approach to affective rating of Chinese words from ANEW. In: D’Mello S, Graesser A, Schuller B, Martin JC, editors. Affective Computing and Intelligent Interaction. Lecture Notes in Computer Science, vol. 6975. Berlin, Heidelberg: Springer; 2011. pp. 121–31.
  88. 88. Schmidtke DS, Schröder T, Jacobs AM, Conrad M. ANGST: Affective norms for German sentiment terms, derived from the Affective Norms for English Words. Behav Res Methods. 2014 Dec;46(4):1108–18 pmid:24415407
  89. 89. Soares AP, Comesaña M, Pinheiro AP, Simões A, Frade CS. The adaptation of the affective norms for English words (ANEW) for European Portuguese. Behav Res Methods. 2012 Mar;44(1):256–69. pmid:21751068
  90. 90. Eilola TM, Havelka J. Affective norms for 210 British English and Finnish nouns. Behav Res Methods. 2010 Feb;42(1):134–40. pmid:20160293
  91. 91. Monnier C, Syssau A. Affective norms for French words (FAN). Behav Res Methods. 2014 Dec;46(4):1128–37. pmid:24366716
  92. 92. Redondo J, Fraga I, Padrón I, Comesaña M. The Spanish adaptation of ANEW (affective norms for English words). Behav Res Methods. 2007 Aug;39(3):600–5. pmid:17958173
  93. 93. Citron FMM, Weekes BS, Ferstl EC. Effects of Valence and Arousal on written word recognition: Time course and ERP correlates. Neurosci Lett. 2013 Jan 15;533:90–5. pmid:23142715
  94. 94. Kloumann IM, Danforth CM, Harris KD, Bliss CA, Dodds PS. Positivity of the English language. PLoS One. 2012;7(1):e29484. pmid:22247779
  95. 95. Mohammad SM, Turney PD, editors. Emotions evoked by common words and phrases: using Mechanical Turk to create an emotion lexicon. Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text; 2010 June; Los Angeles, CA. Association for Computational Linguistics; 2010.
  96. 96. Briesemeister BB, Kuchinke L, Jacobs AM. Discrete emotion effects on lexical decision response times. PLoS One. 2011;6(8):e23743. pmid:21887307
  97. 97. Melzack R, Casey KL. Sensory, motivational, and central control determinants of pain: Anew conceptual model. In: Kenshalo DR, editor. The Skin Senses. Springfield, IL: Charles C. Thomas; 1968. pp.423–43.
  98. 98. Schacht A, Sommer W. Emotions in word and face processing: Early and late cortical responses. Brain Cogn. 2009 Apr;69(3):538–50. pmid:19097677
  99. 99. Kissler J, Herbert C, Peyk P, Junghofer M. Buzzwords: early cortical responses to emotional words during reading. Psychol Sci. 2007 Jun;18(6):475–80. pmid:17576257
  100. 100. Palazova M, Mantwill K, Sommer W, Schacht A. Are effects of emotion in single words non-lexical? Evidence from event-related brain potentials. Neuropsychologia. 2011 Jul;49(9):2766–75. pmid:21684295
  101. 101. Miller GA, Charles WG. Contextual correlates of semantic similarity. Lang Cogn Process. 1991;6(1);1–28.
  102. 102. Gernsbacher MA. Resolving 20 years of inconsistent interactions between lexical Familiarity and orthography, concreteness, and polysemy. J Exp Psychol Gen. 1984 Jun;113(2):256–81. pmid:6242753
  103. 103. Carroll JB, White MN. Age-of-acquisition norms for 220 picturable nouns. Verbal Learning Verbal Behav. 1973;12(5):563–76.
  104. 104. Gilhooly KJ, Logie RH. Age-of-acquisition, imagery, concreteness, familiarity, and ambiguity measures for 1,944 words. Behavior research methods & instruments. 1980;12(4):395–427.
  105. 105. Biemiller A, Rosenstein M, Sparks R, Landauer TK, Foltz PW. Models of vocabulary acquisition: direct tests and text-derived simulations of vocabulary growth. Sci Stud Read. 2014;18(2);130–154.
  106. 106. Brysbaert M. Age of acquisition ratings score better on criterion validity than frequency trajectory or ratings ‘corrected’ for frequency. Q J Exp Psychol. 2017;70;1129–39.
  107. 107. Łuniewska M, Haman E, Armon-Lotem E, Etenkowski B, Southwood F, Darinka A, et al. Ratings of age of acquisition of 299 words across 25 languages: Is there a crosslinguistic order of words? Behav Res Methods. 2016 Sept;48(3);1154–77. pmid:26276517
  108. 108. Morrison CM, Chappell TD, Ellis AW. Age of acquisition norms for a large set of object names and their relation to adult estimates and other variables. Q J Exp Psychol A. 1997;50(3):528–59.
  109. 109. Paivio A. Imagery and verbal processes. New York, NY: Holt, Rinehart and Winston; 1971.
  110. 110. Paivio A, Yuille JC, Madigan SA. Concreteness, imagery, and meaningfulness values for 925 nouns. J Exp Psychol. 1968 Jan;76(1):Suppl:1–25.
  111. 111. Paivio A, Begg I. Imagery and comprehension latencies as a function of sentence concreteness and structure. Percept Psychophys. 1971;10(6):408–42.
  112. 112. Paivio A. Dual coding theory, word abstractness, and emotion: a critical review of Kousta et al. (2011). J Exp Psychol Gen. 2013 Feb;142(1):282–7. pmid:23398183
  113. 113. Schwanenflugel PJ, Shoben EJ. Differential context effects in the comprehension of abstract and concrete verbal materials. J Exp Psychol Learn Mem Cogn. 1983;9(1):82–102.
  114. 114. Schwanenflugel PJ, Harnishfeger KK, Stowe RW. Context availability and lexical decisions for abstract and concrete words. J Mem Lang. 1988;27;499–520.
  115. 115. Rammstedt B, Krebs D. Does Response Scale Format Affect the Answering of Personality Scales? Assessing the Big Five Dimensions of Personality with Different Response Scales in a Dependent Sample. Eur J Psychol Assess. 2007;23:32–38.
  116. 116. Feldman-Barrett L, Russell JA. The structure of current affect: controversies and emerging consensus. Curr Dir Psychol Sci. 1999;8(1):10–4.
  117. 117. Woodforde JM, Merskey H. Some relationships between subjective measures of pain. J Psychosom Res. 1972 Jun;16(3):173–8. pmid:5072910
  118. 118. Ratcliff R, Hockley W, Mckoon G. Components of activation: repetition and priming effects in lexical decision and recognition. J Exp Psychol Gen. 1985 Dec;114(4):435–50. pmid:2934497
  119. 119. Coltheart M, Davelaar E, Jonasson JT, Besner D. Access to the internal lexicon. In: Dornic S, editor. Attention and Performance. New York, NY: Academic Press; 1977. pp. 535–55.
  120. 120. Yarkoni T, Balota D, Yap M. Moving beyond Coltheart’s N: A new measure of orthographic similarity. Psychon Bull Rev. 2008 Oct;15(5):971–9. pmid:18926991
  121. 121. R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna. 2016. Retrieved from https://www.r-project.org/
  122. 122. IBM Corp. R. IBM SPSS Statistics, Version 24.0. Armonk, NY: IBM Corp. 2016.
  123. 123. Rodriguez A, Laio A. Machine learning. Clustering by fast search and find of density peaks. Science. 2014 Jun 27;344(6191):1492–6. pmid:24970081
  124. 124. Rozin P, Berman L, Royzman E. Biases in use of positive and negative words across twenty natural languages. Cogn Emot. 2010;24(3):536–48.
  125. 125. Augustine AA, Mehl MR, Larsen RJ. A positivity bias in written and spoken English and its moderation by personality and gender. Soc Psychol Personal Sci. 2011 Sep;2(5):508–515. pmid:27840670
  126. 126. Garcia D, Garas A, Schweitzer F. Positive words carry less information than negative words. Science. 2012;1(1):3.
  127. 127. Baayen RH. Analyzing linguistic data: a practical introduction to statistics using r. Cambridge University Press; 2006.
  128. 128. Venables WN, Ripley BD. Random and Mixed Effects. In: Modern applied statistics with S. New York, NY: Springer Science; 2002
  129. 129. Borsook D, Becerra L. Emotional Pain without Sensory Pain—Dream On? Neuron. 2009 Jan 29;61(2):153–5. pmid:19186157
  130. 130. Horn C, Blischke Y, Kunz M, Lautenbacher S. Does pain necessarily have an affective component? Negative evidence from blink reflex experiments. Pain Res Manag. 2012 Jan-Feb;17(1):15–24. pmid:22518363
  131. 131. Bird H, Franklin S, Howard D. Age of acquisition and imageability ratings for a large set of words, including verbs and function words. Behav Res Methods Instrum Comput. 2001 Feb;33(1):73–9. pmid:11296722
  132. 132. Juhasz BJ, Lai YH, Woodcock ML. A database of 629 English compound words: ratings of familiarity, lexeme meaning dominance, semantic transparency, age of acquisition, imageability, and sensory experience. Behav Res Methods. 2015 Dec;47(4):1004–1019. pmid:25361864
  133. 133. Stadthagen-Gonzalez H, Davis CJ. The Bristol norms for Age of Acquisition, Imageability, and Familiarity. Behav Res Methods. 2006 Nov;38(4):598–605. pmid:17393830
  134. 134. Wiemer-Hastings K, Krug J, Xu X. Imagery, context availability, contextual constraint, and abstractness. Proceedings of the Annual Meeting of the Cognitive Science Society; 2001 Jan 01; Hillsdale, NJ: Erlbaum; 2001.
  135. 135. Paivio A, Smythe PC, Yuille JC. Imagery versus meaningfulness of nouns in paired-associate learning. Can J Exp Psychol, 1968;22:427–41.
  136. 136. Yee LTS. Valence, arousal, familiarity, concreteness, and imageability ratings for 292 two-character Chinese nouns in Cantonese speakers in Hong Kong. PLoS One. 2017 Mar 27;12(3):e0174569. pmid:28346514
  137. 137. Bonin P, Méot A, Aubert LF, Malardier N, Niedenthal PM, Capelle-Toczek MC. Normes de concrétude, de valeur d’imagerie, de fréquence subjective et de valence émotionnelle pour 866 mots. Annee Psychol. 2003;103(4):655–94.
  138. 138. Guasch M, Ferré P, Fraga I. Spanish norms for affective and lexico-semantic variables for 1,400 words. Behav Res Methods. 2016 Dec;48(4):1358–1369. pmid:26542969
  139. 139. Citron FMM, Weekes BS, Ferstl EC. How are affective word ratings related to lexicosemantic properties? Evidence from the Sussex Affective Word List (SAWL). Applied Psycholinguistics. 2014;35(2):313–331.
  140. 140. Kanske P, Kotz SA. Modulation of early conflict processing: n200 responses to emotional words in a flanker task. Neuropsychologia. 2010 Oct;48(12):3661–4. pmid:20654636
  141. 141. Yao Z, Wu J, Zhang Y, Wang Z. Norms of valence, Arousal, concreteness, Familiarity, imageability, and context availability for 1,100 Chinese words. Behav Res Methods. 2017 Aug;49(4):1374–1385. pmid:27553483
  142. 142. Juhasz BJ. Age-of-acquisition effects in word and picture identification. Psychol Bull. 2005 Sep;131(5):684–712. pmid:16187854
  143. 143. Breivik H, Collett B, Ventafridda V, Cohen R, Gallacher D. Survey of chronic pain in Europe: prevalence, impact on daily life, and treatment. Eur J Pain. 2006 May;10(4):287–333. pmid:16095934
  144. 144. Holmes TH, Rahe RH. The social readjustment rating scale. J Psychosom Res. 1967;11:213–18. pmid:6059863
  145. 145. Baliki MN, Chialvo DR, Geha PY, Levy RM, Harden RN, Parrish TB, Apkarian AV. Chronic pain and the emotional brain: specific brain activity associated with spontaneous fluctuations of intensity of chronic back pain. J Neurosci. 2006 Nov 22;26(47):12165–73. pmid:17122041
  146. 146. Dworkin RH. An overview of neuropathic pain: syndromes, symptoms, signs, and several mechanisms. Clin J Pain. 2002 Nov-Dec;18(6):343–9. Review. pmid:12441827
  147. 147. Fischer A. Gender and emotion: social psychological perspectives. Cambridge University Press; 2000.
  148. 148. Strong J, Mathews T, Sussex R, New F, Hoey S, Mitchell G. Pain language and gender differences when describing a past pain event. Pain. 2009 Sep;145(1–2):86–95. pmid:19586722
  149. 149. Kret ME, De Gelder B. A review on sex differences in processing emotional signals. Neuropsychologia. 2012 Jun;50(7):1211–21. pmid:22245006
  150. 150. Price DD, Harkins SW, Baker C. Sensory-affective relationships among different types of clinical and experimental pain. Pain. 1987 Mar;28(3):297–307. pmid:2952934
  151. 151. Duckworth KL, Bargh JA, Garcia M, Chaiken S. The automatic evaluation of novel stimuli. Psychol Sci. 2002 Nov;13(6):513–9. pmid:12430834
  152. 152. Feldman-Barrett L. Valence is a basic building block of emotional life. J Res Pers. 2006;40(1):35–55.
  153. 153. Larsen RJ, Diener E. Promises and problems with the circumplex model of emotion. In Clark MS, editor. Review of Personality and Social Psychology Vol. 13. Newbury Park, CA: Sage;1992. pp. 25–59.
  154. 154. Watson D, Tellegen A. Toward a consensual structure of mood. Psychol Bull. 1985 Sep;98(2):219–35. pmid:3901060
  155. 155. Witherell D, Wurm LH, Seaman SR, Brugnone NA, Fulford ET. Danger and usefulness effects as a function of concept ancientness. Ment Lex. 2012;7(2):183–209.
  156. 156. Wurm LH. Danger and usefulness: An alternative framework for understanding rapid evaluation effects in perception? Psychon Bull Rev. 2007 Dec;14(6):1218–25. pmid:18229500
  157. 157. Brooks JA, Shablack H, Gendron M, Satpute AB, Parrish MH, Lindquist KA. The role of language in the experience and perception of emotion: a neuroimaging meta-analysis. Soc Cogn Affect Neurosci. 2017 Feb 1;12(2):169–183. pmid:27539864
  158. 158. Lindquist KA, Satpute AB, Wager TD, Weber J, Feldman-Barrett L. The brain basis of positive and negative affect: Evidence from a meta-analysis of the human neuroimaging literature. Cereb Cortex. 2016 May;26(5):1910–1922. pmid:25631056
  159. 159. Kveraga K, Boshyan J, Adams R.B, Mote J, Betz N, Ward N, et al. If it bleeds, it leads: separating threat from mere negativity. Soc Cogn Affect Neurosci. 2015 Jan;10(1):28–35. pmid:24493851