Despite the undisputed role of emotions in teamwork, not much is known about the make-up of emotions in online collaboration. Publicly available repositories of collaboration data, such as Wikipedia editor discussions, now enable the large-scale study of affect and dialogue in peer production.
We investigate the established Wikipedia community and focus on how emotion and dialogue differ depending on the status, gender, and the communication network of the editors who have written at least 100 comments on the English Wikipedia's article talk pages. Emotions are quantified using a word-based approach comparing the results of two predefined lexicon-based methods: LIWC and SentiStrength.
We find that administrators maintain a rather neutral, impersonal tone, while regular editors are more emotional and relationship-oriented, that is, they use language to form and maintain connections to other editors. A persistent gender difference is that female contributors communicate in a manner that promotes social affiliation and emotional connection more than male editors, irrespective of their status in the community. Female regular editors are the most relationship-oriented, whereas male administrators are the least relationship-focused. Finally, emotional and linguistic homophily is prevalent: editors tend to interact with other editors having similar emotional styles (e.g., editors expressing more anger connect more with one another).
Emotional expression and linguistic style in online collaboration differ substantially depending on the contributors' gender and status, and on the communication network. This should be taken into account when analyzing collaborative success, and may prove insightful to communities facing gender gap and stagnation in contributor acquisition and participation levels.
Citation: Iosub D, Laniado D, Castillo C, Fuster Morell M, Kaltenbrunner A (2014) Emotions under Discussion: Gender, Status and Communication in Online Collaboration. PLoS ONE 9(8): e104880. doi:10.1371/journal.pone.0104880
Editor: Pablo Brañas-Garza, Middlesex University London, United Kingdom
Received: December 20, 2013; Accepted: July 17, 2014; Published: August 20, 2014
Copyright: © 2014 Iosub et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: These authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Emotions are the glue of human societies  (cf. , p.27) and their significant influence on human behavior is undisputed. Online collaborative communities, as increasingly important social spaces for teamwork and self-expression, make no exception and are permeated by emotions . Yet, little is known about the emotional “ecosystem” of online collaborative endeavors, and our work contributes to this knowledge gap.
As field of study we choose the English Wikipedia, one of the largest peer-production communities, which provides an excellent case scenario, given its size and importance. The conversations in the Wikipedia discussion pages are especially valuable. These pages represent arenas of cooperation and conflict between users (to whom we will refer to as editors from now on) with the goal of improving encyclopedic content.
Conversation is essential for coordination in such spaces , and therefore facilitates fruitful collaboration and successful content creation. Like in any other human collaborative experience, communication also triggers emotions and breeds particular emotional environments that may influence teamwork in the short and long run.
To analyze the emotional expression and communication style used in Wikipedia discussions we utilize two established word-counting measures and differentiate according to status and gender. Since each lexicon highlights different aspects of emotion and language, their concurrent use provides us with a rich understanding of editor interactions.
Indisputably, differences in status also reflect differences in language use , and the communication context in which emotions play out becomes interesting to investigate. Consequently, our first research question analyzes how emotional expression and communication style in online collaboration differ according to status. We expect to find differences in emotional expression according to the status of Wikipedia editors since research on emotion in social structures suggests that emotional expression is a significant marker of status in the social hierarchy . Analyzing emotions in this context may prove insightful considering the slow-down in editor growth from 2006 onwards .
Our analyses bring insight into the emotional profile of higher status contributors which is important since the emotions of individuals higher in rank are contagious for followers and can impact the performance of the entire social structure . These contributors bear therefore the responsibility to channel their emotions towards constructive ends for the community.
Similarly, gender has been found to explain diverse facets of language and emotion in offline settings , hence we expect Wikipedia male editors to display a different communication and emotional mode compared to female ones. The second focus of our research is, therefore, on how the emotional expression and communication style of editors depend on gender. The differences are worth exploring, considering that the gender gap (i.e., a strong inequality in the gender distributions of the participants) is a serious source of concern for this collaborative community .
In our discussion of the first two research questions we refer to a crucial concept in the leadership literature – relationship-oriented communication , a type of communication focused on establishing and maintaining social ties . We expect to find evidence that Wikipedia editors exhibit features of this kind of speech, and we analyze online discussions to uncover status and gender differences in its use.
Finally, the network of editors and messages in Wikipedia can provide interesting evidence for phenomena such as “emotional congruence” or “emotional homophily” in a peer-production context. We investigate them as third and fourth research questions. More exactly, we pursue a brief analysis of Wikipedia's interaction networks based on messages and replies exchanged between editors to examine how editors' emotions relate to the emotions of the editors they reply to during interaction (emotional congruence), and how the emotions of editors are related to those of the editors they interact more frequently with (emotional homophily). These phenomena have been observed in online fora  and blogs ; they refer to the similarity of comment-reply pairs regarding expressed emotion – emotionally-loaded dialogue is followed by replies with higher emotional content, whereas neutrality is met with neutrality – and to the principle of “birds of a feather flock together” – users tend to interact with others expressing similar emotions.
The following section presents a short overview of related research in diverse fields such as computer-science, psychology and communication, structured according to the research focus.
Communication on Wikipedia
The largest human encyclopedia ever written, Wikipedia, is one of the most prominent examples of successful online collaboration to date. In fact, considering the thousands of failed online collaboration efforts , its size and success are quite miraculous. This noteworthy performance has motivated a flurry of research activity  on topics ranging from leadership behaviors to motivations to contribute.
Communication is quintessential to Wikipedia. The complex interaction system allows for persistent social interaction among participants, which facilitates the emergence, maintenance and continuous redefinition of social structures and collective goals . Editors frequently discuss changes to articles on article talk pages, while user talk pages (“personal” pages) resemble a personal wall and function as a public mail inbox . Both types of pages are used for interaction, but the conversation in article talk pages is article-focused, quite “formalized and policy driven”  and meant to ensure article quality, while discussion on personal talk pages is editor-focused . In light of the different emphasis of Wikipedia talk pages, our research sheds light on how this translates into individual patterns of emotion and communication.
Emotions and peer-production
A recent contribution on emotions in the online collaborative world is presented in . Similar to the present paper, the study presents an investigation of emotions in Wikipedia depending on gender and status. The analysis is based on the ANEW (Affective Norms for English Words) lexicon and shows that women express more positive emotion than men, while higher-status editors are more positive compared to lower-status ones. We draw heavily on the research presented in  by using it as a starting point to compare and contrast our results regarding emotions. On the other hand, we diverge significantly from it, since our investigation allows for a richer and more fine-grained analysis at both the emotion and communication level. In particular, the word-counting measures we use facilitate the understanding of emotion within the communication context.
Earlier work on the relationship between emotions and online contribution behavior can be traced back to Joyce and Kraut , who identified no relationship between receiving positive feedback and the likelihood of posting again in a public newsgroup. Simply receiving feedback was enough to increase contribution behavior, irrespective of the valence (i.e., positive and negative affect) of the response.
On the other hand, subsequent research found clear evidence that feedback valence impacts participation behavior. For example, Cheshire and Antin  investigated the effect of response on contribution behavior, and found that positive feedback such as gratitude (e.g., receiving automatic thank you replies) increased the users' number of solved puzzles.
Similarly, Wang, Kraut and Levine  reported that emotional support in the form of caring messages increased the commitment to an online health support group more so than informational support. In fact, informational support alone was associated with a greater risk of drop-off from the community, whereas the combination of emotional and informational support was the most effective for continued participation in the community.
A study especially relevant to this research was undertaken by Zhu, Kraut and Kittur  on the effect of leadership styles on contribution behavior in Wikipedia. The authors focused on the outcome of task-oriented and relationship-oriented communication. Task-oriented communication is characterized by assertive, directive and instrumental speech such as directive statements, information provision, as well as critical evaluation of contributions . On the other hand, relationship-oriented communication is affiliative and includes expression of support, agreement, and acknowledgment of others' contributions . The authors found that relationship-oriented messages increased the probability of subsequent edits. On the contrary, negative feedback decreased members' contributions greatly. Messages coming from higher-status were more influential compared to those coming from lower-status members.
A recent study by Kucuktunc et al.  investigated emotions in a large-scale Q&A community depending on factors such as gender, age or experience in the community. Their findings suggest that women are more emotional and express more positive emotion compared to men, while more experienced users give increasingly neutral answers.
There is also an interesting stream of literature on the network properties of emotions. For example, Chee  finds in an online health forum that those who communicate often have similar emotion levels, while Thelwall  reports the same conclusion for MySpace. This is consistent with either a homophily hypothesis (“Birds of a feather flock together”) or with an emotional contagion explanation. The present work briefly investigates this phenomenon from a network analysis perspective in a novel, peer-production context, without aiming at disentangling the two competing explanations.
Finally, with regard to Wikipedia, previous studies have investigated mixing patterns in this community from different perspectives, to assess whether there is a tendency of editors to interact with similar others (assortativity). Diversity according to experience and volume of activity ,  has been observed as a distinguishing characteristic of interactions in the community: expert editors tend to interact preferentially with newbies and less active editors, and vice-versa. On the other hand, assortative mixing patterns have been observed with respect to other markers of identity, such as gender: females tend to interact more with other females, and males with other males . From a social identity perspective, a recent study highlighted that Wikipedians tend to communicate more with others supporting the same political party in personal talk pages, while no preference was observed in article talk pages . Emotional expression and style of editors have also been found to drive their interactions, with a preference for communication with editors having a similar style ; here we deepen this analysis by taking into account also discrete emotions and linguistic styles of editors.
Conversational and emotional markers of status and gender
Human language contains markers of status, and social hierarchy can be understood through and mapped by analyzing language , . Hence, in this work we will compare the power relation in Wikipedia on several dimensions of written expression, and expect to find salient differences, also with regard to emotional aspects.
First, previous literature on communication differences between the powerful versus the weak indicates that those lower in status use more tentative speech as a signal of insecurity , such as maybe, could, should. We expect to find such differences between high-status Wikipedia editors (whom we will refer to as administrators or admins) and regular editors.
Moreover, there is much interest in the literature with regard to the significance of first-person singular pronouns. The evidence is mixed, however: whereas the higher use of first-person active pronouns (e.g., “I”) can commonsensically be viewed as a signal of power or empowerment , a number of text analyses with the LIWC lexicon – find the opposite, i.e. heightened use of “I” is associated with the lower status of the writer. This self-focus has been interpreted as a strategy to draw attention to the less-powered self, in an attempt to highlight one's importance and merits. Taking gender into account, while an earlier study by Mulac, Bradac and Gibbons  finds evidence for men's use of first singular pronouns, more recent studies such as Mehl and Pennebaker  and Newman and colleagues  find that females are more self-focused. We study the use of “I” pronouns for the Wikipedia community, and propose several explanations for the results.
Despite being a showcase for open collaboration and peer-production, Wikipedia is not an open-source setting with a flat hierarchy. In fact, some authors contend that a strong motivation to contribute to Wikipedia is gaining and exercising power in the community , . Higher-status Wikipedia editors are called administrators or admins. They are a special category of Wikipedia editors who generally assume responsibility in the community, which is associated with privileges compared to regular editors. For example, they are allowed to delete pages or block other editors .
There is little research on the emotional and language differences depending on status in Wikipedia. Panciera et al.  find that gaining status in Wikipedia leads to an increased formalization of speech. For example, higher-status editors refer more often to Wikipedia policies during discussion. On the other hand, they maintain a positive attitude, while less-experienced editors express more negative emotion . We add to this literature by providing a much more detailed account of the differences in emotion and language use between administrators and regular editors.
This paper also examines gender differences in terms of emotional and linguistic expression. Previous literature finds a robust marker of gender – the positivity of language. Women use more positive language, and particularly so when engaged in conversation. Mehl and Pennebaker  found these results in a naturalistic conversation setting, and Newman et al.  found that women express more positive emotion when interacting with others, but not in other contexts, e.g., essays or stream of consciousness (a stream of consciousness writing task asks participants to write down thoughts and feelings). Additionally, Kivran-Swaine and colleagues  found that in Twitter women disclose more positive emotion than men, and especially so when interacting with other women. Given previous research, we expect Wikipedian women to use more positive language compared to male editors.
Furthermore, women are more likely than men to practice relationship-oriented speech to positively engage with others. A meta-analysis conducted by Leaper and Ayres  finds that women use significantly more affiliative speech than men. However, the difference is statistically significant only for conversations on non-personal topics, self-disclosures, and deliberations, as well as for same-gender groups (but not for mixed-gender groups). Men, on the other hand, use significantly more task-oriented, assertive speech than women. Again, the effect is significant for same-gender groups, and for discussions of non-personal topics or deliberations. Since the interaction in Wikipedia is generally on non-personal topics, yet takes place in a mixed-gender setting, we do not have a precise theoretical expectation concerning gender differences in relationship-oriented speech.
Previous literature is not clear on whether there is a gender advantage in the expression of negative emotion. While some authors find that women express more negative emotions than men , , others (e.g. ) report the opposite.
Finally, we investigate gender differences in the expression of anger. Previous research indicates that societal expectations discourage women from showing anger , , and that men have an advantage in the use of anger words . While Newman and colleagues  do not find evidence for this for all types of text, they do so for conversations, suggesting that the gender difference in anger expression is activated particularly during social interactions. Consequently, we expect that women will use fewer anger words compared to men.
Women are severely under-represented in the Wikipedia community, leading to an important gender gap. The participation rate for female editors is a subject of contention, ranging from a mere 9%  to 16.1% . Its low level has been attributed to women's conflict aversion and sensitivity to criticism, as well as to a lack of confidence in their expertise and contribution value . Given that 22% of female editors have reported disagreeable interactions and receiving inappropriate messages , our investigation of emotions and dialogue could prove insightful for the gender gap issue.
Materials and Methods
In this section we describe the dataset consisting of Wikipedia discussions on article and personal talk pages, as well as the methodology used for the analysis of emotions and language.
For comparison purposes we make use of the same dataset used in . Only the conversation history of the most active editors in discussion pages was retained from a complete snapshot of the English Wikipedia . This consists of the editors who have written at least 100 comments during discussions on article talk pages; since not all editors have a personal page, this reduced the dataset to 11637 editors for the personal talk pages. Table 1 reports the basic statistics of the data.
The dataset contains information on editors' status (administrator versus regular editor) and gender, which allows comparison across different editor groups. While information on editor status is available through the Wikipedia API, collecting gender information is less straightforward and can prove challenging. In this case gender identification was possible using a combination of methods, ranging from using Wikipedia's API to crowdsourcing the gender identification task to Crowdflower (see  for more details). Table 2 summarizes the editor sample classified according to status and gender.
We measure the emotional content of comments in article talk pages and user talk pages with a lexicon-based method consisting of two established word-counting measures - the Linguistic Inquiry and Word Count (LIWC)  and SentiStrength . The simultaneous use of several types of lexicons allows us to: (i) cross-validate the results of our and previous analyses, given that emotional valence is a common characteristic of lexicons; (ii) offer a rich understanding of interactions, since each lexicon highlights different aspects of emotion and language.
Not least, we aim to compare and contrast our results with those obtained using ANEW (Affective Norms for English Words) and shown in . For a meaningful comparison, we first selected only those comments from the discussions in the articles and personal talk pages for which at least one ANEW word was found. This decision was done to restrict the possible influence of the article and discussion topics, as reported in , on the results. This sampling strategy resulted in a database of more than 7.4 million comments for the article talk pages, and around 3 million comments for the personal pages, which were then used for the computation of the LIWC and SentiStrength scores restricting us on the subset of these comments written by our set of editors.
There are slight differences across the lexicons with regard to the definition of valence for each comment:
- LIWC gives a positive and a negative score representing the percentage of positive(negative) words in a comment ,
- SentiStrength provides a positive and a negative score based on the most positive(negative) sentence in a comment ,
- while ANEW provides a single score on a scale from 1 (extremely negative) to 9 (extremely positive) .
Regardless of these differences, LIWC and SentiStrength results by and large validate and confirm the ANEW findings of  regarding valence, as we will explain in detail in the Results section. Moreover, our lexicon comparison allows us to bring additional understanding to the Wikipedian landscape, particularly with regard to gender- and status-related differences in language use. The following paragraphs describe in more detail the characteristics of the three lexicons, as well as the procedural steps we applied for the text- and sentiment analysis of Wikipedia content.
The Linguistic Inquiry and Word Count (LIWC) was developed by the psychologists Pennebaker, Booth and Francis in the early 90s to automatize psychological analysis of written expression, and is now the preferred automated sentiment detection method in psychology, while also gaining ground in the computer sciences. Moreover, the LIWC lexicon  allows us to identify language differences between Wikipedia editor groups that go beyond emotion expression.
We aggregated all comments of a single editor (comments posted by editors on article talk pages) in one file, which was then analyzed with LIWC. For the personal pages we analyzed with LIWC the entire content of an editor's talk page, and discriminated between messages received and written by an editor on her own talk page.
LIWC provides two scores for basic emotion: positive valence and negative valence. Positive (negative) valence in LIWC is defined as the percentage of positive (negative) words  in a text, in this case as the percentage from the total number of words written by an editor during her Wikipedia activity (which takes values between 0 and 100).
We also analyzed the comments with respect to the many different groups of measures that LIWC provides – e.g. relationship-orientation, temporal orientation or certainty of the written expression. This allowed us to bring additional insight to the differences in emotion and language use between the Wikipedia editor groups.
These additional LIWC measures, including personal pronouns such as “I” or “you”, several discrete measures of emotions (e.g., anger, anxiety and sadness) or social words, are derived in a similar fashion to the LIWC basic emotions, as the percentage of words from a given category compared to the total number of words written by an editor during her Wikipedia activity. The LIWC dictionary for anger contains words such as annoyed, argh or bastard; the one for anxiety, stressed, terrifying, uneasy, etc.; while the measure for sadness includes agony, depressed or grief. The group of certainty measures refers to the use of tentative words (anyhow, depends, doubt), certainty words such as guaranteed or obviously, as well as filler words, e.g., oh well, i mean. Temporal orientation refers to the use of verbs in past, present or future tense. Finally, the dictionary for social words is composed of words referring to family, friends and humans in general, e.g., reply, daughter, baby. The percentage of social words together with the percentages of personal pronouns of different types define the group of relationship-orientation metrics. An overview of the LIWC measures with dictionary sizes and further examples can be found in Table 3. We also performed a lexicon validation for the categories relationship-orientation, certainty and anger, which lead accuracy levels situated around 0.70. Details about the results for these three LIWC categories can be found in Text S1. To our knowledge the first two categories have not been validated before.
This is a very recent word-counting tool , and is considered the state-of-the-art lexicon method for sentiment detection in short web texts . Based on the 2007 revised version of LIWC, SentiStrength also accounts for modes of textual expression specific to the online environment, e.g. emoticons and abbreviations.
SentiStrength also provides a positive and a negative score for emotional valence. It is an adapted version of LIWC, much more appropriate for social media analysis. We analyze sentiment at comment level and then average across them for each editor. The SentiStrength emotion score is calculated at the sentence level, and then summarized at the comment level. At the sentence level, SentiStrength detects the number of positive and negative words. At the comment level, SentiStrength offers two different ways to compute the summarization:
- based on the mode of the sentence scores (e.g., if the most frequently encountered positive score for the sentences in a comment is 2, then the comment receives a score of 2)
- based on the strongest positive and negative emotion expressed in a comment (e.g., if the maximum positive score for the sentences in a comment is 2, then the comment receives a score of 2).
We chose to discard the SentiStrength scores based on the mode of the emotional value of the sentences, given that a problematic situation arises when the distribution of scores is multi-modal. Moreover, most of academic research conducted with SentiStrength focuses on the results based on the maximum valence words.
To increase the results comparability of the different lexicons, we also ran an ANEW-weighted version of SentiStrength (SentiStrength scores weighted by the number of ANEW words found in the comment from which the score is derived, so that each sentence has the same importance with the two lexicons). In the paper we report the non-weighted scores, since the same results hold for the weighted version of the scores. Consequently, the Results section below contains only this form of score computation, under the subheading “SentiStrength”.
For comparison purposes with , we present below a short description of the lexicon. The Affective Norms for English Words (ANEW) is a list of words with emotional scores for valence collected from human raters on a scale from 1 to 9 . As opposed to SentiStrength and LIWC, most words in the ANEW list are not feeling-related words, i.e., they do not directly reference emotion (such as happy or sad); instead, they cover the entire spectrum of valence (including neutrality), and describe concepts that trigger associated emotions.
In the following we use the example comments from  to illustrate the differences between the three lexicons. The marked comments in Table 4 can be understood as follows: bold-faced words have been identified in the SentiStrength library, words in italics are part of the LIWC dictionaries, while underlined words have been found in the ANEW lexicon. Should a word be both bold and in italic (e.g., challenge), this signifies that it has been found both in the SentiStrength and LIWC lexicons, and so on.
We observe a high degree of overlap between the three lexicons, in terms of recognized words. However, the lexicons differ in terms of the scores assigned to the words. For example, the second comment includes the words “sex”, “lover” and “war”, and is therefore highly polarized/ambivalent regarding sentiment. The comment is composed of 44 words, out of which 3 are marked as positive in the LIWC lexicon (“well” and two times “lover”) and 2 as negative (two times “war”) which translates into a positive LIWC Score of and a negative score of . When using SentiStrength we detect the words with the strongest positive and negative emotions. For the example SentiStrength correctly indicates that the sentence contains both high positive emotion (+4), caused by the word “lover”, and negative emotion (-3), caused by the word “war”; as does LIWC. For comparison we also indicate the the valence level in ANEW, which loses information through averaging (5.5) over the disperse emotional scores associated to the words (indicated via underlining) of the ANEW lexicon. This illustrates, among others, the lexicons' similarities and complementarities.
To analyze the network characteristics of emotions in Wikipedia, we computed scores for “emotional congruence” (similarity level of message-reply pairs) and “emotional homophily” (similarity of editors' emotional profiles). While emotional congruence is derived from the network of exchanged messages, homophily is based on a network of editors constructed by the rule that the editors included in the network have exchanged at least one reply. Emotional congruence is calculated as the average of the difference between the score of each comment and the score of the comment to which it replies; for this analysis all comments were considered, not just those coming from our set of highly active discussants. Meanwhile, to measure homophily, after aggregating comments at editor-level, assortativity is computed in the network using the shuffle test, as explained in detail below.
We first assessed the normality of the distributions of our variables of interest using Kolmogorov-Smirnov tests of normality. Since most variables are not normally distributed despite the large sample size, we examine the differences between Wikipedia editor groups by computing two-tailed Mann-Whitney U-tests which have a greater efficiency than t-tests on non-normal distributions. To increase the readability of tables, we highlight differences that are significant with by showing the corresponding p-value in bold, and underlining the larger of the two population averages. For the cases (marked with an asterisk *) in which the averages were not informative, we include the mean ranks and underline the larger value. The sample size for the tests differs depending on whether the analysis was conducted on article or personal talk pages (considering that not all editors have a personal page and not all editors that have a personal page write or receive messages). However only 445 (6%) of the editors in our sample do not have a personal talk page. This percentage is even lower for administrators or for editors whose gender we were able to identify and can be neglected as possible explanation for the differences we observe in our analyses. The samle size also differs whether the analysis was conducted with LIWC or SentiStrength (for a number of editors SentiStrength could not classify the messages as strongly positive or negative). An overview of the test samples can be found in Table S1.
We also use one-tailed sign-tests (against the hypothesis that the differences are either larger or smaller than 0) to assess the significance of the differences observed when measuring emotional congruence.
Assortativity with shuffle test.
To measure assortativity in the network according to a certain variable of the nodes, we compute the correlation () between the value of this variable for each pair of connected nodes, and we then perform a shuffle test . This test is based on the comparison of the network with a number (in our case 100) of randomized equivalents, i.e. networks in which nodes have the same characteristics and the same number of connections (degree) as in the original, but links between nodes are randomly shuffled. Significance of the observed pattern is thus measured via the z-score, , where is the correlation coefficient for the desired variable in the original network, while and are, respectively, the average and the standard deviation of the same correlation measured in the randomized networks.
Results and Discussion
We compare editors across two dimensions: status (Admins versus Non-admins) and gender (Male versus Female), and two lexicons: LIWC and SentiStrength. While we do not repeat the ANEW results presented in , we frequently address them as a means of cross-validation. We compare across both article talk pages and personal (“user talk”) pages. We conducted a correlation analysis between the per editor metrics of their comments in article and personal talk pages and found low correlations (at maximum of 0.35). This suggests that the editors' speech differs in the two spaces and justifies their separate analysis. The section ends with a network-level similarity assessment of the emotions expressed by editors who interact with one another.
Emotions and Status
We first investigate differences in emotion and language according to status in Wikipedia. The LIWC results for the article talk pages shown in Table 5 suggest that admins express, on average, more positive emotion than regular editors. The result is significant also for the personal talk pages (see Table 6), suggesting that the positive attitude extends to the more private sphere of personal pages. Moreover, both in article and personal pages administrators refrain from using negative emotion, and the comments they receive also contain less negative emotion.
These findings relate well to De Choudhury et al.'s  results on affect at the workplace. Their study focuses on messages exchanged via an internal microblogging tool at a large company, and shows that managers use more positive and less negative language when conversing with regular employees. In our case, we provide similar evidence for this phenomenon in a self-organized online collaborative environment.
Finally, the SentiStrength analysis adds interesting nuances to the results. It suggests that in article and personal pages (Tables 5 and 6) non-admins have a “higher pitch” when expressing both positive and negative emotion. Therefore, when they use positive and negative emotion words, they use stronger ones than the admins. Moreover, our analyses show (see right columns of Table 6) that regular editors receive stronger positive and negative words on their personal pages compared to admins, indicating that the “higher pitch” may be reciprocated.
When comparing admins and non-admins in terms of the discrete emotions they express, a distinctive picture emerges. The LIWC analysis suggests that regular editors are, on average, more emotional than admins. In article talk pages they express more affect, and in particular more anxiety, anger and sadness compared to admins (Table 5). The lower emotionality in administrators' communications is corroborated by the findings based on ANEW, reported in , that admins use less emotional content, as suggested by the lower arousal. Administrators' neutrality along with the increased referencing of Wikipedia policies  may be an expression of administrators' higher task-orientation, i.e., focus on setting goals and accomplishing tasks.
The results with regard to discrete negative emotions are mixed for the personal talk pages (Table 6). Administrators refrain from expressing negative emotion in general, but this does not hold for two particular emotions: anxiety and sadness. Similarly, the comments admins receive also contain more anxiety and sadness. This finding suggests that administrators are more relaxed with the expression of emotion within the “private” spaces of personal talk pages, while being impersonal in article talk pages.
The fact that administrators receive more anxiety and sadness (but not anger) on their personal pages may suggest a possible “ingratiation” strategy . Anxiety and sadness are signals of personal vulnerability , , and coupled with the lack of anger, may indicate submissiveness towards the admins. Therefore, the fact that admins also receive more positive emotion and overall less negative emotion, could be interpreted as part of an ingratiation strategy , i.e., regular editors may wish to come across as attractive or likeable to higher status Wikipedians. A similar strategy has been documented by Danescu-Niculescu-Mizil et al. , who report that regular editors of Wikipedia change their linguistic style to match that of administrators. We provide evidence that, apart from language, emotion can also be used to characterize power imbalance in a collaborative environment.
To conclude, admins are more neutral in communication than regular editors in the public space of article talk pages. This is in line with previous literature , , which finds a tendency for increasing emotion neutrality and formalization as editors gain more experience in the community. When admins express emotion, they do so in a moderately positive manner. Our findings support the ones with ANEW reported in , and go in accordance with the idea that administrators generally have a positive tone and wish to embody the Wikipedian spirit of collaboration characterized by “good faith” . Meanwhile, non-admins are more effusive in their emotional expression.
Dialogue and Status
With the goal of bringing additional understanding to the differences in emotionality between admins and non-admins, we took advantage of the numerous text analyses that LIWC provides and compared Wikipedia editors across several dimensions of linguistic expression reported in Table 7 and 8, to understand how the differences can be understood in light of the status relations existent in the community.
Compared to the rather impersonal comments of admins, regular editors connect more to other people. In article discussions and (to a somewhat lesser degree) in personal walls, they make more other-references (more personal pronouns) and use more words related to the social domain, such as mentions of friends and family.
Together with the poignant emotional expression of regular editors, this is a traditional marker for a socio-emotional speech style – non-admins think about the people in the conversation and try to establish a close relationship between themselves and the audience, whereas administrators are more socially detached. Our findings are in contrast with the results of Zhu et al.  who find that both admins and regular editors engage predominantly in task-oriented communication, i.e. communication focused on goal setting and task accomplishment . This may be due to the very different conceptualization of task-oriented versus relationship-oriented language in their paper.
Further research is needed to determine whether the preoccupation with others is a consequence of the need to obtain the approval of higher status Wikipedians, and whether this is associated with higher levels of ingratiation and overaccomodation, as per Dino, Reysen and Branscombe . In Wikipedia the community life is important and the prospect of being evaluated by peers (e.g., in order to attain administrator status) may motivate editors to maintain good relationships with higher authority community members.
In line with previous literature on the speech insecurity of the powerless , we find that non-admins are less confident in article talk pages. They are generally more preoccupied with the topic of certainty, and use more tentative words (e.g., perhaps, maybe), and more filler words (e.g., errr, hmmm). We find little evidence of speech insecurity in personal pages: non-admins use fewer tentative and filler words compared to admins. This suggests that regular editors communicate more confidently within their personal space.
Admins are more focused on the future, while regular editors seem more concerned with the present (and in article talk pages, also with the past). This may be related to a more pragmatic attitude of administrators, especially interested in the actions to be undertaken and “getting things done”. Considering also the more neutral tone of administrators, the interest in the future may be a reflection of their higher task-orientation, characterized by directive and instrumental speech.
Interestingly, regular editors are more insecure only during discussion on article talk pages, indicative of the power imbalance within the public space of article discussion; for the personal pages this result does not hold, i.e. editors use more self-assured language within their personal space confirming the results from a question-answering system where experienced contributors were found to give more neutral answers , our detailed analyses of language suggests in wide agreement that admins tend, indeed, towards neutrality – they “rule with reason”, are more formal and impersonal in their discussion of Wikipedia articles compared to regular editors, while generally keeping the tone positive. Future research would be needed to assess to what extent the positive tone of administrators could in fact be due to sarcasm , .
In contrast, regular editors are characterized by a socio-emotional, people-oriented speech style, possibly as a means of ingratiation. Especially on article talk pages they are emotional and personal, reference others more, express more negative emotions, and are more effusive with all basic emotions by using stronger emotions words. Regular editors are also more concerned with the past and the present, while admins show a more pragmatic interest in the future.
We suggest that a potential explanation for the observed differences in emotion and language use could be administrators' tendency towards task-orientation compared to regular editors' leaning towards relationship-orientation within the Wikipedia community. This is an important distinction, considering that Zhu, Kraut and Kittur  investigated leadership behaviors across all levels of hierarchy in Wikipedia and found that task-oriented leadership had a mixed effect on contributions (with the transactional component being the most effective and the aversive one being the most detrimental), while relationship-oriented leadership had the strongest positive effect on contributions. Future research should determine how the communication style of higher status editors influences growth and, possibly, stagnation in online collaboration.
Finally, in an organizational setting, De Choudhury et al.  interpreted the positive tone of higher-status employees (managers and executives) as a manifestation of transformational leadership, a leadership style characterized by the commitment to inspire, excite and maintain high motivation levels in workers, e.g. by communicating an uplifting vision for the future . Our study finds that positive emotion is only one piece of the puzzle, and future research is needed to indicate whether positive emotion may in fact co-occur with task-oriented communication.
Emotions and Gender
In the following, we extend our analyses and examine gender differences in terms of emotional expression and language, aiming to bring much needed insight to the gender gap issue in Wikipedia editorship and in ICT, in general. Previous research with ANEW suggests that women express higher valence than men in article talk pages . In the following we focus on the results from the LIWC and SentiStrength analyses, which characterize written expression on more than just valence.
The LIWC lexicon (Tables 9 and 10) suggests that women indeed express more positive emotion than men during discussion both in article and personal talk pages. The results obtained using SentiStrength follow a similar pattern – women display more high-pitched valence, i.e., they use words that have a stronger positivity than men. This is in line with previous literature, regarding both the online and offline context , , , . However, the above found differences might disappear when taking into consideration the topics on which women and men choose to work on, since women prefer topics discussed in a markedly positive manner . Indeed, research on gender language differences has only recently considered the topic of conversation. Laniado et al.  provide indication that there are no gender differences in emotional expression when controlling for topic, but further research in a natural setting is needed to shed light on this issue.
The difference in positive emotion remains significant when comparing men and women at the administrator level with both LIWC and SentiStrength, thereby contradicting the results obtained with ANEW . On the other hand, female administrators are similar to male administrators with respect to all other types of emotion in article pages (Table 11). For personal talk pages female admins appear more emotional than male admins for both sent and received messages, suggesting that women restrict their emotionality to a certain extent in the public space of article talk (Table 12).
It must be said, however, that male administrators are found to be more positive than regular male editors (Table 13) in the article talk pages, possibly indicating that they promote a positive working environment, pivotal to the good functioning of the Wikipedia community. On the other hand, male admins express less emotion compared to regular editors. In fact, they differ significantly from the regular counterparts on almost all basic and discrete emotion dimensions with respect to article discussions. We do not observe this result for the personal pages, suggesting that male administrators adhere to administrator role expectations only within the public space of article discussions.
Lastly, our analysis with LIWC shows that women non-admins express more negative emotions, anxiety and anger in particular, than women admins during conversation on article and personal pages. This comes in contrast with the results of  who find that women non-admins have similar emotional profiles to women admins. On the other hand, SentiStrength suggests a slightly different pattern for the article talk pages – women non-admins are more effusive in the expression of strong positive emotion, while being similar with regard to strong negative emotion expression, thereby confirming the findings of Laniado et al.  (Table 13).
In contrast to the mixed results in previous literature , , , our analyses find no significant gender difference in negative emotional expression, which is the case for both article (Table 9) and personal talk pages (Table 10). This holds for both LIWC and SentiStrength in terms of overall negative emotion, as well as in particular for anxiety and sadness.
In contrast with previous literature , , which suggests that women express less anger, we do not observe this effect, neither for article nor for personal talk pages. Regarding male anger expression we observe that male admin comments in article talk pages contain less anger than those of male non-admins, which corroborates our previous results of administrator neutral tone. Moreover, male admins receive less anger in personal pages, supporting the ingratiation hypothesis we presented previously.
All lexicons find unanimously that women express more positive emotion compared to men, which might in fact be owing to topic choice of male versus female editors, as observed in . More novel is that male administrators are quite different from male normal editors in terms of emotional expression.
The fact that administrators (in particular, male administrators) are more neutral, more formal (they reference Wikipedia policies more often) and less concerned with other Wikipedia editors could indicate that they are more task-oriented. On the other hand, regular editors (females, in particular) are clearly more relationship-oriented than their higher-status counterparts.
Dialogue and Gender
In the previous section we have compared the emotional expression of male and female editors in Wikipedia talk pages. Further on, we increase our understanding of gender differences by characterizing editor style along other dimensions of linguistic expression, as done for status.
One of our most robust findings is that women and men differ in terms of the preoccupation with the social domain (see Tables 14 and 15), consistent with the observation that women are more interested in relating to others, i.e., in building and maintaining relationships. Women are more sociable compared to men, since they use more socially-related words, and make more self- and other-references, e.g., use of more personal pronouns. The personal pages of women are also found to be more social than those of men, and in particular here other-referencing is more common.
We therefore add to the rather undisputed literature on women's' rapport interest , ; on the other hand, it is unexpected to find this result for the article talk pages, i.e., in a mixed-gender, non-personal topic context, contrary to the meta-analysis of Leaper and Ayres . Even more revealing is that this finding holds across hierarchy levels in the community. Table 16 illustrates the strength of relationship-orientation across different editor groups. Specifically, we find that female regular editors are more people-focused than male regular editors, and the same holds for female administrators versus male administrators. Finally, female regular editors are more concerned with relationships than female admins, and similarly for male regular editors compared to male admins.
All in all, regular editors are more relationship-focused than administrators, and women more than men. The least relationship-oriented are male administrators, while women regular editors are the most people-focused. Interestingly, this holds true only for the article talk pages. For the personal talk pages the only similar finding is that female admins are more concerned with other editors than male admins. We interpret the difference between article and personal pages in light of the public-private space dichotomy. While “at work” administrators adopt a more neutral, formal and instrumental speech in discussion of non-personal topics on article talk pages, but this difference attenuates in the more private sphere of personal pages.
To support our assumption of a possible link between relationship-orientation and a personable approach towards others, we conducted a qualitative analysis of 100 comments from the article talk pages. We find evidence for the co-occurrence of the two interaction styles. Comments high in relationship-orientation are also high in “niceness”, reveal a genuine interest in understanding other editors' perspectives and generally indicate a collaborative attitude. The classification of the comments high in relationship-orientation according to content yields several types of comments: 1. inviting comments that explain the edit in a friendly tone, and call for further intervention and collaboration; 2. common perspective-building comments that are focused on understanding others and solving debates in a constructive manner; 3. appreciative comments that contain positive emotions and celebrate others' actions. This suggests that relationship-orientation may be conducive to successful collaboration and further research should shed light on this issue.
An additional finding is that women self-reference more than men in the public space of article talk pages by using “I” and “I”-related pronouns. A possible explanation could be that women are more insecure, and express themselves using first person singular and uncertainty words (e.g. I guess, maybe), which is more compliant . We find some evidence for this in our data, as we detect differences between men and women with regard to the use of certainty words in article talk pages and filler words in personal talk pages (see Tables 14 and 15).
Another potential explanation relates to the minority status that women have in Wikipedia. The psychological literature suggests that feeling distinctive within a group (e.g. being part of a minority) predicts self-focused attention , . Therefore, if women Wikipedians are aware that they constitute a minority, this could trigger more self-focus.
Most likely, the use of the active first-person singular is a signal of heightened social sensitivity, expressing consciousness of the subjectivity of one's own point of view. By saying phrases such as “I think”, “I believe” and by expressing positive emotions, women may be engaging in relationship-oriented speech. Consequently, the use of “I” may be an indicator that they are considering perspectives other than their own – a personable approach to relating to others.
We find minor differences between men and women with regard to the use of (un)certainty words (certainty words in article talk pages and filler words in personal talk pages - see Tables 14 and 15). This is an unexpected result, given that women's' lack of confidence in their ability to edit has been suggested as a possible cause for the gender gap in Wikipedia . This does not appear to be the case for women highly active and involved in discussion (those in our sample, who pass the 100 comment-mark).
Female editors are more concerned with the past and present. However, this finding holds only for article talk pages.
One robust finding is that women are more interested in relating to others than men – they use more social words and reference themselves and others more. This is true at each hierarchy level in the community. Therefore, the use of more relationship-oriented language could help attract and retain more women to Wikipedia. Moreover, our qualitative analysis suggests that the higher relationship-orientation is connected to an open and constructive attitude towards collaboration, while Zhu, Kraut and Kittur  show that it increases contributions. Since women are more relationship-oriented, attracting them would create a positive circle and lead to a more inviting peer-production environment, ultimately triggering increased community morale and more contributions.
Network characteristics of emotion and language
So far we have studied how emotions and language vary according to characteristics of the editors, namely status and gender. To do this, we have considered each comment separately, and we have then aggregated all comments of (or directed to) the same editor. In this section, on the contrary, we disregard individual editor characteristics, but we consider the relationships between comments and their replies to investigate how emotion and language relate to editor interactions. In particular, we investigate this at the comment level, studying how individual messages differ from the comments they reply to (emotional congruence) and at the editor level, studying how similar are the emotional profiles of editors who interact with one another (emotional homophily).
According to the ANEW results reported in , editors tend to reply with significantly () higher valence. The analyses with LIWC on the same dataset, including about 2.5 million comment-reply pairs, confirm this finding and contribute to its understanding. Here we use a non-parametric sign-test which does not assume any particular features of the tested distribution. In Table 17 we report the results of the sign-test on the differences between the metrics of comment and reply pairs. The table shows whether replies are more likely to contain larger or lower values of each metric; more specifically, the column “Overlay” indicates the difference between the number of cases when replies have a higher or a lower value of the corresponding metric; i.e, it indicates in which proportion it is more likely that replies have a higher value for that metric (or less likely, in case of a negative percentage).
We confirm  and find a surplus of replies that contain a higher percentage of positive words (4.3%) and consistently also an under-representation of replies that contain a higher percentage of negative words, although to a lesser extent (−1.3%). Both results are highly significant ().
Accordingly, replies tend to contain more emotional content (affect), but less anger, sadness and anxiety than the messages they reply to, confirming a general trend to keep a positive tone in discussions. Moreover, it is more likely that replies contain more first and second personal pronouns, especially “You” (10.5%), and more socially-oriented words (See Table 17). Together with the expression of positive emotion, these are all markers of a socio-emotional communication style which appears to characterize editors when replying.
Interestingly, SentiStrength results for congruence contrast with LIWC and ANEW results, and suggest that while more positive on average, replies tend to contain slightly less “high-pitched” positive emotions, and more high-pitched negative emotions.
By investigating homophily we can determine whether the editors that interact (exchange at least one message on article talk pages) have similar emotional and conversational profiles. According to the concept of assortativity , assortative networks are characterized by a preference to interact with similar others; meanwhile, in disassortative networks dissimilar people are more connected. We find two types of assortativity:
Assortativity according to emotions.
Confirming the results shown for the ANEW variables in , the reply network is assortative for all emotion variables, with values of always greater than 2 (Table 18). Therefore, editors are more likely to interact with others having a similar emotional style. This is a confirmation of emotional homophily in the community, also for discrete emotions measured by the LIWC lexicon such as anger, anxiety or sadness.
Figure 1 illustrates the finding above for assortativity in terms of anger expressed in the messages. It depicts the network of replies, where editors are connected if they exchanged at least ten messages. Node colors range from blue for editors low in anger expression, to red for editors who use a high proportion of words expressing anger. One can visually observe that similar editors tend to “stick together”, i.e., connect more than dissimilar editors.
The color of each node depends on the proportion of words expressing anger in the comments written by the corresponding editor, from blue (low) to red (high). Two editors are connected if they exchanged at least 10 replies in article talk pages. Node size is proportional to the number of connections.
Assortativity according to other speech-related LIWC variables.
Interestingly, as shown in Table 19, the reply network is largely assortative also with respect to all other language use variables considered in this study, indicating homophily also in terms of relationship-orientation, certainty and temporal orientation. Therefore, editors with similar emotional and conversational profiles tend to connect more in Wikipedia.
While we do not report the results here, our analysis of the communication network for personal talk pages presents very similar patterns – the network is clearly assortative for all variables. In this analysis we considered the network of messages written by editors on one another's talk page, but still characterizing each editor with her style in article discussions. In this way we looked at personal communications in a network that is not based on the comments on which the scores are computed, so we could expect a different outcome. Instead, the results confirm assortativity for all variables, suggesting that editors who express emotion and communicate similarly on article talk pages also interact more with one another on personal pages.
We find conclusive evidence for emotional homophily, a relatively widespread phenomenon in community sites , , : editors tend to interact preferentially with others having a similar style, and expressing similar emotions. As noted in previous literature , there are two main explanations for this phenomenon: the first is that similar editors are more likely to interact with one another because they have similar interests, and the second is emotional contagion. It is of course difficult to separate these two phenomena, which are probably both present. However, our finding that editors having a similar style in article talk pages also communicate more with one another in personal spaces suggests that there is something more than just emotional contagion at the micro-level, and that indeed birds of a feather flock together. Interestingly, none of the observed emotional and linguistic features of speech makes an exception to this rule.
At the message-reply level, the results seem to confirm the finding of  of a general trend to exhibit a positive attitude when replying to other editors, in line with the community guidelines and netiquette (for example ). In fact, editors overall tend to reply with more positive emotion, more relationship-oriented language and more affect, but less anger, which can be seen as signs of an increased attention towards the others when replying. However, replies tend to contain more high-pitched negative words, which suggests a more complex picture of interactions at a micro-level; a more fine grained analysis, based on qualitative observation, could shed further light on this aspect, while a possible effect of negative replies on editor participation could be investigated by studying editor activity over time.
Our results provide insights for the understanding of community evolution and engagement, and have implications for communities facing membership stagnation, similar to Wikipedia.
We find that higher-status editors promote a neutral, impersonal and more formal conversation tone in Wikipedia. They “rule with reason”, and maintain a mildly positive tone – a crucial aspect to the good functioning of the collaborative project. Nevertheless, it is not clear whether administrator neutrality fulfills the needs of the community in the long run. Peer-production communities are settings of voluntary contribution, and emotions play an important role in group dynamics, requiring expression. Relationship-oriented communication has been found to increase contributions  and, interestingly, regular editors use this linguistic style more than administrators. Consequently, the tone of group moderators, and more generally the interaction spaces of such communities should be adapted to facilitate both positive exchanges and the venting out of negative emotion in a constructive manner.
For this aim, the role of female editors is paramount. Indeed our analyses (both automatic and the brief manual analysis of content) provide strong evidence that female editors engage in relationship-oriented speech that is conducive to a positive working environment. Interestingly, this result holds also for female administrators, who diverge significantly from male administrators by being more relationship-oriented. By increasing the diversity of leadership styles and by promoting an atmosphere of openness and concern for others, women leaders play a pivotal role in such online spaces.
These results have implications also for the gender gap issue. Together with the finding of  that women tend to interact preferentially with other women, our results suggest that being able to involve more women and to give them more space in the community would also result in a virtuous cycle of female participation, through the creation of a communication environment where they feel more comfortable.
Furthermore, we identify a special group of Wikipedian women who look confident in their own abilities. Previous research suggested lack of self-confidence as a possible reason for the gender gap in Wikipedia . This is not the case for women who are active discussants in the community, and who exhibit the same level of confidence as men, irrespective of their status in Wikipedia.
Beyond gender, our results indicate that the discussion network is highly assortative also with respect to emotion and communication style. Editors communicate more with others having a similar style, both in terms of emotional and communication profiles. This suggests that clusters of different emotions and linguistic styles can be identified and managed in collaborative communities.
Finally, we find a relation between hierarchy and emotional and linguistic patterns. Regular editors use more insecure expressions than administrators on article talk pages, but not on personal pages. This suggests that there is a perceived sense of hierarchy in Wikipedia (and possibly intensified by administrators' formal tone in communication). Further research should indicate to what extent and under which conditions a perceived hierarchy is beneficial to the long-term well-being of a collaborative community.
To conclude, this paper increases our understanding of peer-production processes, in terms of the emotional expressions and responses of contributors. We have provided data and insights about the emotional and conversational dimension of Wikipedia, and how emotions and language use are related to the profiles of editors and to their interactions.
Further lines of research include conducting a similar multi-metrics study within other communities, over a long time period, and considering also less active and anonymous editors in Wikipedia. Although we already relied on a combination of quantitative and qualitative approaches (including crowdsourcing to assess the validity of automatic techniques), a richer usage of human annotations and one that accounts for non-textual emotion (e.g., emoticons, “barn stars”, and virtual gifts) would certainly help to get a deeper and more fine-grained understanding of the results, as well as to provide valuable input to increase the performance of automatic methods. For example, it would be important to be able to detect in which contexts messages with high-pitched positive emotions are likely to be just sarcastic.
We would also like to incorporate a time dimension to study the effects of emotions. For instance, related to Wikipedia's decline in membership levels, it would be interesting to test if the editors that eventually abandon the community received more negative messages than the editors that stay and get more engaged. It would be also useful to analyze the vocabulary of the community, similar to , to investigate the relationship between the editor involvement and the adhesion to the community lexicon. Moreover, the emotion and communication metrics (e.g., positive emotion or socio-emotional style) could be linked to performance metrics of the collaborative effort, such as the quality of Wikipedia articles.
Lexicon validation. A brief review of previous validation research of emotional lexicons and a description of our own efforts in this direction.
Sample sizes for the Mann-Whitney tests for article and personal talk pages.
Conceived and designed the experiments: DI DL CC MFM AK. Performed the experiments: DI DL. Analyzed the data: DI DL AK. Wrote the paper: DI DL CC MFM AK.
- 1. Durkheim E (1893) The Division of Labor in Society. New York: Free Press 1956.
- 2. Collins R (1990) Stratification, emotional energy and the transient emotions. In: Kemper T, editor. Research Agendas in the Sociology of Emotions. New York: Suny Press. pp. 27–57.
- 3. Benkler Y (2006) The wealth of networks: how social production transforms markets and freedom. New Haven, CT: Yale Univ. Press. Available: http://www.benkler.org/Benkler_Wealth_Of_Networks.pdf. Accessed 28-July-2014.
- 4. Kittur A, Kraut RE (2010) Beyond Wikipedia: coordination and conflict in online production groups. In: Proceedings of the 2010 ACM conference on Computer supported cooperative work. New York, NY, USA: ACM, CSCW '10, pp.215–224. doi:10.1145/1718918.1718959.
- 5. Dino A, Reysen S, Branscombe N (2009) Online interactions between group members who differ in status. Journal of Language and Social Psychology 28: 85–93. doi: 10.1177/0261927x08325916
- 6. http://stats.wikimedia.org. Accessed 22-July-2013.
- 7. Keltner D, Van Kleef G, Chen S, Kraus M (2008) A reciprocal influence model of social power: Emerging principles and lines of inquiry. Advances in Experimental Social Psychology 40: 151–192. doi: 10.1016/s0065-2601(07)00003-2
- 8. Brebner J (2003) Gender and emotions. Personality and Individual Differences 34: 387–394. doi: 10.1016/s0191-8869(02)00059-4
- 9. Wikimedia Foundation (2011). Editor survey. Available: http://meta.wikimedia.org/wiki/Editor_Survey_2011. Accessed 22-July-2013.
- 10. Bass B (1990) Bass & Stogdills Handbook of Leadership; Theory, research, and managerial applications, 3rd ed. New York: Free Press.
- 11. Bales R (1958) Task roles and social roles in problem-solving groups. In: Maccoby EE, Newcomb TM, Hartley EL, editors, Readings in social psychology, 3rd ed. New York: Holt, Rinhart, & Winston: Guilford Press. pp. 437–447.
- 12. Chee B (2010). Sickness and health: Homophily in online health forums. Available: https://www.ideals.illinois.edu/handle/2142/14926. Accessed 28-July-2014.
- 13. Thelwall M (2010) Emotion homophily in social network site messages. First Monday 15. doi: 10.5210/fm.v15i4.2897
- 14. Hill B (2013) Almost wikipedia: What eight early online collaborative encyclopedia projects reveal about the mechanisms of collective action. Working Paper.
- 15. Okoli C, Mehdi M, Mesgari M, Nielsen Fr, Lanamäki A (2012) The people's encyclopedia under the gaze of the sages. SSRN eLibrary. Available: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2021326. Accessed 28-July-2014.
- 16. Forte A, Lampe C (2013) Defining, understanding, and supporting open collaboration: Lessons from the literature. American Behavioral Scientist 57. doi: 10.1177/0002764212469362
- 17. Laniado D, Tasso R, Volkovich Y, Kaltenbrunner A (2011) When the Wikipedians talk: Network and tree structure of Wikipedia discussion pages. In: ICWSM-11 - 5th International AAAI Conference on Weblogs and Social Media. pp. 177–184.
- 18. Viegas FB, Wattenberg M, Kriss J, van Ham F (2007) Talk before you type: Coordination in wikipedia. In: Proceedings of the 40th Annual Hawaii International Conference on System Sciences. HICSS ’07, pp. 78–87. doi:10.1109/HICSS.2007.511.
- 19. Hara N, Shachaf P, Hew KF (2010) Cross-cultural analysis of the Wikipedia community. JASIST 61: 2097–2108. doi: 10.1002/asi.21373
- 20. Laniado D, Castillo C, Kaltenbrunner A, Morell M (2012) Emotions and dialogue in a peer production community: the case of Wikipedia. In: Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration. WikiSym ’12, pp. 9 1–9: 10. doi: 10.1145/2462932.2462944
- 21. Joyce E, Kraut RE (2006) Predicting continued participation in newsgroups. Journal of Computer-Mediated Communication 11. doi: 10.1111/j.1083-6101.2006.00033.x
- 22. Cheshire C, Antin J (2008) The social psychological effects of feedback on the production of internet information pools. Journal of Computer-Mediated Communication 13: 705–727. doi: 10.1111/j.1083-6101.2008.00416.x
- 23. Wang YC, Kraut R, Levine JM (2012) To stay or leave?: The relationship of emotional and informational support to commitment in online health support groups. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. New York, NY, USA: ACM, CSCW ’12, pp. 833–842.
- 24. Zhu H, Kraut R, Kittur A (2012) Effectiveness of shared leadership in online communities. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. New York, NY, USA: ACM, CSCW ’12, pp. 407–416.
- 25. Hersey P, Blanchard K (1981) So you want to know your leadership style? Training & Development Journal 36.
- 26. Bowers D, Seashore S (1966) Predicting organizational effectiveness with a four-factor theory of leadership. Administrative Science Quarterly 11: 238–263. doi: 10.2307/2391247
- 27. Kucuktunc O, Cambazoglu BB, Weber I, Ferhatosmanoglu H (2012) A large-scale sentiment analysis for Yahoo! answers. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining. WSDM ’12, pp. 633–642. doi:10.1145/2124295.2124371.
- 28. Laniado D, Tasso R (2011) Co-authorship 2.0: Patterns of collaboration in Wikipedia. In: Proceedings of the 22Nd ACM Conference on Hypertext and Hypermedia. New York, NY, USA: ACM, pp. 201–210.
- 29. Neff JJ, Laniado D, Kappler KE, Volkovich Y, Aragn P, et al. (2013) Jointly they edit: Examining the impact of community identification on political interaction in wikipedia. PLoS ONE 8: e60584. doi: 10.1371/journal.pone.0060584
- 30. Erickson B, Lind EA, Johnson BC, O’Barr WM (1978) Speech style and impression formation in a court setting: The effects of “powerful” and “powerless” speech. Journal of Experimental Social Psychology 14: 266–279. doi: 10.1016/0022-1031(78)90015-x
- 31. Ng S, Bradac J (1993) Power in language: Verbal communication and social influence. Sage Publications.
- 32. Weintraub W (2003) Verbal behavior and personality assessment. In: Post J, editor, The psychological assessment of political leaders with profiles of Saddam Hussein and Bill Clinton, Ann Arbor: University of Michigan Press. pp. 137–153.
- 33. Hancock JT, Beaver DI, Chung CK, Frazee J, Pennebaker JW, et al. (2010) On lying and being lied to: A linguistic analysis of deception. Behavioral Sciences of Terrorism and Political Aggression, Special Issue: Political Activism 2: 108–132. doi: 10.1080/19434471003597415
- 34. Scholand A, Tausczik Y, Pennebaker J (2010) Social language network analysis. In: Proceedings of the 2010 ACM conference on Computer supported cooperative work. New York, NY, USA: ACM, CSCW '10, pp. 23–26.
- 35. Chung C, Pennebaker J (2007) The psychological functions of function words. In: Fiedler K, editor, Social Communication, New York: Psychology Press. pp. 343–359.
- 36. Mulac A, Bradac J, Gibbons P (2001) Empirical support for the gender-as-culture hypothesis. Human Communication Research 27: 121–152. doi: 10.1111/j.1468-2958.2001.tb00778.x
- 37. Mehl M, Pennebaker JW (2003) The sounds of social life: A psychometric analysis of students' daily social environments and natural conversations. Journal of Personality and Social Psychology 84: 857–870. doi: 10.1037/0022-3522.214.171.1247
- 38. Newman M, Groom CJ, Handelman LD, Pennebaker JW (2008) Gender differences in language use: An analysis of 14,000 text samples. Discourse Processes 45: 211–246. doi: 10.1080/01638530802073712
- 39. Farrell H, Schwartzberg M (2008) Norms, minorities, and collective choice online. Ethics & International Affairs 22: 357–367. doi: 10.1111/j.1747-7093.2008.00171.x
- 40. Goldspink C (2008) Social self-regulation in computer mediated communities: The case of wikipedia. Unpublished manuscript.
- 41. Danescu-Niculescu-Mizil C, Lee L, Pang B, Kleinberg J (2012) Echoes of power: language effects and power differences in social interaction. In: Proceedings of the 21st international conference on World Wide Web. New York, NY, USA: ACM, WWW ’12, pp. 699–708.
- 42. Panciera K, Halfaker A, Terveen L (2009) Wikipedians are born, not made. In: Proceedings of the ACM 2009 International Conference on Supporting Group Work. GROUP ’09, pp. 51–60. doi:10.1145/1531674.1531682.
- 43. Yasseri T, Sumi R, Kertesz J (2012) Circadian Patterns of Wikipedia Editorial Activity. PLoS ONE 7: e30091. doi: 10.1371/journal.pone.0030091
- 44. Kivran-Swaine F, Brody S, Diakopoulos N, Naaman M (2012) Of joy and gender: emotional expression in online social networks. In: Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work Companion. New York, NY, USA: ACM, CSCW ’12, pp. 139–142.
- 45. Leaper C, Ayres M (2007) A meta-analytic review of gender variations in adults' language use: Talkativeness, affiliative speech, and assertive speech. Personality and Social Psychology Review 11: 328–363. doi: 10.1177/1088868307302221
- 46. Thomson R, Murachver T (2001) Predicting gender from electronic discourse. British Journal of Social Psychology 40: 193–208. doi: 10.1348/014466601164812
- 47. Mulac A, Studley LB, Blau S (1990) The gender-linked effect in primary and secondary students' impromptu essays. Sex Roles 23: 439–469. doi: 10.1007/bf00289762
- 48. Heilman M (2001) Description and prescription: How gender stereotypes prevent women's ascent up the organizational ladder. Journal of Social Issues 57: 657–674. doi: 10.1111/0022-4537.00234
- 49. Rudman L (1998) Self-promotion as a risk factor for women: The costs and benefits of counterstereotypical impression management. Journal of Personality and Social Psychology 74: 629–645. doi: 10.1037//0022-35126.96.36.1999
- 50. Hill BM, Shaw A (2013) The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation. PloS one 8: e65782. doi: 10.1371/journal.pone.0065782
- 51. Collier B, Bear J (2012) Conict, criticism, or confidence: an empirical examination of the gender gap in wikipedia contributions. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work. CSCW ’12, pp. 383–392. doi:10.1145/2145204.2145265.
- 52. Wikipedia:Database (2010) Available: http://en.wikipedia.org/wiki/Wikipedia:Database_download. Accessed March-2010.
- 53. Pennebaker J, Chung C, Ireland M, Gonzales A, Booth R (2010) The development and psychometric properties of liwc2007. Technical report, LIWC.net, Austin, TX. Available: http://www.liwc.net. Accessed 28-July-2014.
- 54. Thelwall M, Buckley K, Paltoglou G, Cai D, Kappas A (2010) Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology 61: 2544–2558. doi: 10.1002/asi.21416
- 55. Bradley M, Lang P (1999) Affective norms for English words (ANEW). Technical Report C-1, The Center for Research in Psychophysiology, University of Florida.
- 56. Pfitzner R, Garas A, Schweitzer F (2012) Emotional divergence influences information spreading in twitter. In: ICWSM-12 - 6th International AAAI Conference on Weblogs and Social Media. The AAAI Press, pp. 543–546.
- 57. Anagnostopoulos A, Kumar R, Mahdian M (2008) Influence and correlation in social networks. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD ’08, pp. 7–15.
- 58. De Choudhury M, Counts S (2013) Understanding affect in the workplace via social media. In: Proceedings of the 2013 conference on Computer supported cooperative work. New York, NY, USA: ACM, CSCW ’13, pp. 303–316. doi:10.1145/2441776.2441812.
- 59. Barrett K, Campos J (1987) Perspectives on emotional development ii: A functionalist approach to emotions. In: Osofsky J, editor. Handbook of infant development. New York: Wiley. pp. 555–578.
- 60. Izard C, Ackerman B (2000) Motivational, organizational and regulatory functions of discrete emotions. In: Lewis M, Haviland JM, editors. Handbook of Emotions. New York: Guilford Press. pp. 253–264.
- 61. Reagle J (2010) Good Faith Collaboration: The Culture of Wikipedia. Cambridge, MA: MIT Press.
- 62. Zhu H, Kraut RE, Wang YC, Kittur A (2011) Identifying shared leadership in wikipedia. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, CHI ’11, pp. 3431–3434. doi:10.1145/1978942.1979453.
- 63. Davidov D, Tsur O, Rappoport A (2010) Semi-supervised recognition of sarcastic sentences in twitter and amazon. In: Proceedings of the Fourteenth Conference on Computational Natural Language Learning. Stroudsburg, PA, USA: Association for Computational Linguistics, CoNLL ’10, pp. 107–116.
- 64. Tsur O, Davidov D, Rappoport A (2010) Icwsm–a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In: ICWSM-10 - 4th International AAAI Conference on Weblogs and Social Media. pp. 162–169.
- 65. Dodds B (1997) Does the transactional-transformational leadership paradigm transcend organizational and national boundaries? American Psychologist 52: 130–139. doi: 10.1037/0003-066x.52.2.130
- 66. Schwartz HA, Eichstaedt JC, Kern ML, Dziurzynski L, Ramones SM, et al. (2013) Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE 8: e73791. doi: 10.1371/journal.pone.0073791
- 67. Mulac A, Lundell TL (1994) Effects of gender-linked language differences in adults' written discourse: Multivariate tests of language effects. Language & Communication 14: 299–309. doi: 10.1016/0271-5309(94)90007-8
- 68. Mullen B (2008) Operationalizing the effect of the group on the individual: A self-attention perspective. Journal of Experimental Social Psychology 45: 211–246. doi: 10.1016/0022-1031(83)90025-2
- 69. Wegner D, Schaefer D (1978) The concentration of responsibility: An objective self – awareness analysis of group size effects in helping situations. Journal of Personality and Social Psychology 36: 147–155. doi: 10.1037/0022-35188.8.131.52
- 70. Newman M (2003) Mixing patterns in networks. Physical Review E 67: 026126. doi: 10.1103/physreve.67.026126
- 71. Wikipedia:Talk Page Guidelines (2014) Available: http://en.wikipedia.org/wiki/Talk_page_guidelines. Accessed on 21-07-2014.
- 72. Danescu-Niculescu-Mizil C, West R, Jurafsky D, Leskovec J, Potts C (2013) No country for old members: User lifecycle and linguistic change in online communities. In: Proceedings of the 22nd International Conference on World Wide Web. WWW ’13, pp. 307–318.