Understanding confounding effects in linguistic coordination: an information-theoretic approach

We suggest an information-theoretic approach for measuring linguistic style coordination in dialogues. The proposed measure has a simple predictive interpretation and can account for various confounding factors through proper conditioning. We revisit some of the previous studies that reported strong sig- natures of stylistic accommodation, and find that a significant part of the observed coordination can be attributed to a simple confounding effect - length coordination. Specifically, longer utterances tend to be followed by longer responses, which gives rise to spurious correlations in the other stylistic features. We propose a test to distinguish correlations in length due to contextual factors (topic of conversation, user verbosity, etc.) and turn-by-turn coordination. We also suggest a test to identify whether stylistic coordination persists even after accounting for length coordination and contextual factors.


Introduction
Communication Accommodation Theory [1] states that people tend to adapt their communication style (voice, gestures, word choice, etc.) in response to the person with whom they interact. Originally, experiments on linguistic accommodation were confined to small scale laboratory settings with a handful of participants. The recent proliferation of digital (or digitized) communication data offers an opportunity to study nuances of human communication behavior on much larger scales. A number of recent studies have indicated presence of stylistic coordination in communication [2][3][4][5], where one person's use of a linguistic feature (e.g. prepositions) increases the probability that a response will include the same feature. Linguistic style coordination (or matching) has been used to predict relationship stability [2] and negotiation outcomes [6], understand group cohesiveness [3], and infer relative social status and power relationships among individuals [5].
Most reports of linguistic style coordination have been based on correlational analysis. Thus, such claims are susceptible to various confounding effects. For instance, it is known that there is significant length coordination in dialogues, in the sense that a longer utterance from user Y tends to solicit a longer response from user X [7]. Thus, if the probability of an utterance containing a feature, e.g. prepositions or words whose second letter is "r", depends only on length, this will create the illusion of stylistic coordination on the given feature.
Here we attempt to remedy the problem and propose an information-theoretic framework for characterizing stylistic coordination in dialogues. Namely, given a temporally ordered sequence of utterances (verbal or electronic statements depending on the context) by two individuals, we characterize their stylistic coordination with time-shifted mutual information. The proposed coordination measure characterizes the dependence between the stylistic features of the original post and the response. In addition, we provide a computational framework to account for confounders when measuring stylistic coordination.
We revisit some of the case studies where linguistic coordination was reported and demonstrate that a significant part of the observed correlations in linguistic features can indeed be explained by length coordination rather than stylistic accommodation. In particular, most stylistic features that exhibit statistically significant correlation exhibit little to no correlation after length coordination has been taken into account.
We also focus on the observed length correlations, and examine whether it is due to turnby-turn coordination between the participants, or can be attributed to other contextual factors. We construct a statistical permutation test and demonstrate unequivocally that turn-by-turn length coordination in dialogues indeed takes place. Finally, we develop a similar test for stylistic features, and demonstrate that at least for one of the datasets, the remnant coordination (after conditioning on length) cannot be explained by contextual factors alone and has to be due to turn-by-turn level coordination between the speakers.

Measuring Stylistic Coordination Representing Stylistic Features
To represent stylistic features in utterances, we use Linguistic Inquiry Word Count (LIWC) [8], which is a dictionary-based encoding scheme that has been used extensively for evaluating emotional and psychological dimensions in various text corpora. The latest version of the LIWC dictionary contains around 4500 words and word stems. Each word or word stem belongs to one or more word categories or subcategories. Various LIWC categories include positive and negative emotion, function words, pronouns, articles, and so on. Here we focus on eight LIWC categories that have been used in previous studies [5]: articles, auxiliary verbs, conjunctions, high-frequency adverbs, impersonal pronouns, personal pronouns, prepositions, and quantifiers. Utterances are represented as eight-component binary vectors indicating the presence or absence of each linguistic marker [5].

Information-theoretic measure of coordination
Each dialogue is a sequence of utterance exchanges between two participants. Following [4,5] we binarize the stylistic features of utterances, so that a dialogue is represented as fo m k ; r m k g K k¼1 , where o m k ; r m k ¼ f0; 1g indicate the absence or presence of the stylistic marker m, and K is the total number of exchanges in a dialogue. Since we focus on coordination between the same stylistic markers, we will drop the superscript m from now on. We use the convention O to represent the originator-the person who is producing the original utterance in a single exchange, R to represent the respondent-the person who is replying to the originator.
Let p(o, r) be the joint distribution of the random variables O and R. We characterize the amount of stylistic coordination using Mutual Information (MI) [9]; see S1 File for a brief overview of basic information theoretic concepts: is the Shannon entropy of O, and H(OjR) is the entropy of O conditioned on R. Note that in our case the arguments are temporarily ordered: O is always the initial utterance, and R is the response, so that Eq 1 in fact defines time-shifted mutual information. Thus, even though MI is symmetric with respect to its argument, the coordination between two users may be asymmetric. Recall that mutual information between two variables measures the average reduction in the uncertainty of one variable, if we know the other variable. Thus, in essence, the proposed measure of stylistic coordination quantifies how the use of a marker m in an utterance of O's can help to predict R's usage of m in the immediate response. In contrast to linear correlation measures, mutual information is well suited for handling strongly non-linear dependencies.
We measure the correlation between two variables after conditioning on a third variable, Z, via Conditional Mutual Information, defined as IðO : RjZÞ ¼ HðOjZÞ À HðOjR; ZÞ: ð2Þ Below we will use CMI to account for the confounding effect of the utterance length by conditioning on it. Namely, the actual stylistic accommodation, after accounting for the length coordination, is given by I(O : RjL R ), where L R is the length of the utterance by user R.

Estimating mutual information from data
Given a set of samples fo k ; r k g K k¼1 , our goal is to estimate mutual information between O and R. We could do this by first calculating the empirical distribution p(o, r) and then using Eq 1. However, it is known that this naive plug-in estimator tends to underestimate the entropy of a system. Instead, here we use the statistical bootstrap method introduced by DeDeo et al. [10], which attempts to reduce the bias of the naive estimator by estimating a bootstrap correction term. The estimate of bias comes from comparing the entropy of the empirical distribution to estimates of entropy from several bootstrap datasets drawn randomly according to the empirical distribution. See [10] for more details.
While the above discrete estimator works well for evaluating mutual information between discrete stylistic variables, it is not very useful for evaluating mutual information between two length variables, due to limited number of samples we have. Instead, we will use a continuous estimator introduced by Kraskov et al. [11]. This non-parametric estimator searches the knearest neighbors to each point, and then average the mutual information estimated from the neighborhood of each point. It has been shown that this estimator is asymptotically unbiased and consistent. Discussion of different entropy estimators can be found in [12] and references therein.

Length As A Confounding Factor
We applied our coordination measures to two datasets previously studied in [5]: oral transcripts from the Supreme Court hearings, and discussion among Wikipedia editors. In the Supreme Court Data, there are 11 Judges and 311 Lawyers conversing with each other. We obtain 51,498 utterances from all the dialogues among 204 cases. In the Wikipedia dataset, users are classified into two categories, Administrators, or Admins, and non-Admins. All of the users interact with each other on Wikipedia talk pages, where they discuss issues about specific Wikipedia pages. We focus on dialogues where each participant make at least two exchanges within a dialogue, which results in over 30,000 utterances.
Ideally, we would like to calculate linguistic accommodation between any pair of individuals O and R who have participated in a dialogue. Unfortunately, most pair-wise exchanges are rather short and do not produce sufficient samples for evaluating mutual information or conditional mutual information. Instead, we group the individuals according to their roles, and then use aggregated samples to calculate stylistic coordination between the groups. The groups correspond to Judges and Lawyers for the Supreme Court data, and Admins and non-Admins for the Wikipedia data. The bias in estimators for conditional mutual information and mutual information are generally different. Therefore, rather than estimating mutual information directly, we use a conditional mutual information estimator where we condition on randomly permuted values for L R . We repeat this procedure for four hundred times to produce 99% confidence intervals for I(O : R) (blue bars). The green bars give the 99% confidence intervals in case there is no stylistic coordination by estimating CMI with R's utterances permuted (erasing any stylistic coordination).
The blue dots show the mutual information between the corresponding stylistic features, and suggest strong linguistic correlations between the groups. This effect, however, is strongly diminished after conditioning on the length of utterances (red dots). For instance, the coordination scores on features Impersonal Pronoun, Article, and Auxiliary Verb are reduced by factors of * 6.7, * 4.8, and 5.3, respectively, after conditioning on length. For the feature Conjunction, the 99% confidence interval of coordination score is above the confidence interval of zero information before conditioning, and falls into the confidence interval of zero information after conditioning. Similarly, in Fig 1(b), the coordination scores for five out of eight markers (Impersonal Pronoun, Article, Adverb, Preposition, Quantifier) become practically zero after conditioning, suggesting that the observed coordination in those stylistic features are due to length correlations.
A similar picture holds for the Wikipedia dataset shown in Fig 2. Again, we observe nonzero mutual information in all the features. However, this correlation is significantly diminished after conditioning on length. In fact, both non-admins coordinating to admins (Fig 2(a)) and admins coordinating to non-admins (Fig 2(b)) have an extremely weak signal after conditioning on length (all below 0.005). In particular, for non-admins coordinating to admins (Fig  2(a)), the red dots of five out of eight features lie in the zero conditional mutual information confidence interval. For these five features in Fig 2(b), we cannot rule out the null hypothesis that all stylistic coordination is due to phenomenon of length coordination.
Another interesting observation is that there is significant asymmetry, or directionality, in stylistic coordination. For instance, by comparing Fig 1(a) and 1(b) we see that the mutual information is significantly higher from lawyers to judges than vice versa. A similar (albeit less pronounced) asymmetry is present for the Wikipedia data as well. This type of asymmetry has been used to suggest that the relative strength of stylistic accommodation reflects social status [5]. However, Figs 1 and 2 illustrate that the asymmetry is drastically weakened after conditioning on length (red dots), suggesting that the phenomenon of higher stylistic coordination from lawyers to judges (and from non-admins to admins for the Wikipedia dataset) is due to the confounding effect of length. Unfortunately, a direct assessment of this effect in a single conversation is not feasible due to the insufficient number of utterances for calculating conditional mutual information. Nevertheless, in S2 File we suggest a different approach for addressing the above problem, and find that asymmetry in stylistic coordination can be explained by asymmetry in length coordination.
To conclude this section, we note that some of the correlations in stylistic features persist even after conditioning on length. One can ask whether this remnant correlation is due to turn-by-turn level linguistic coordination, or can be attributed to other confounding factors. We address this question in detail later in the text.

Understanding Length Coordination
As discussed in the previous section, the observed correlations in linguistic features can be attributed to coordination in the length of utterances. Here we analyze this phenomenon in more detail. In particular, we are interested whether the observed length correlations are due to turn-by-turn coordination, or can be attributed to other contextual factors. For instance, consider a scenario that in one conversation, Alice and Bob are always conversing using short statements, while in another conversation they exclusively use long statements, perhaps due to different topics of conversation. Length coordination is found if data from these two conversation is aggregated, however, this coordination only reflects Alice's and Bob's response to the topic of conversation. More generally, aggregating data might lead to effects similar to Simpson's paradox [13].
To understand the possible extent of various confounding factors (we call them contextual factors), consider the Bayesian network model that incorporates both contextual factors and length coordination, as shown in Fig 3. Here L O and L R are random variables representing the length of an utterance by the originator O and the respondent R, respectively. In the model with both solid and dashed lines in Fig 3(b), L O explicitly influences L R . While if we only have the soild lines in Fig 3(a), L R is independent of L O after conditioning on the context C. Thus, the model in Fig 3(a) assumes that there is only contextual coordination while Fig 3(b) implies turn-by-turn coordination. Note that in principle, the contextual factor C can vary within a single conversation, for example, the theme of a conversation may change as time goes by. But for simplicity, we will assume that the contextual factor C does not change within the dialogue or conversation.

Information-theoretic characterization of length coordination
A direct measure of Turn-by-turn Length Coordination (TLC) is given by the following conditional mutual information: Additionally, we define the Overall Length Coordination (OLC) as Thus, OLC captures not only the length coordination in a turn-by-turn level, but also the confounding behaviors between L O , L R and C. In fact, OLC can be decomposed into two items: The second item of right hand side in Eq 5 indicates the multivariate mutual information (MMI) (also known as interaction information [14] or co-information [15]), and characterizes the amount of shared information between L O , L R and C.
A straightforward method to test for turn-by-turn coordination is to evaluate TLC described in Eq 3. Indeed, L O and L R are conditionally independent of C if and only if TLC = 0. However, direct evaluation of TLC is not possible due to the lack of sufficient number of samples, e.g., the number of exchanges within a specific dialogue. Nevertheless, it is possible to test the turn-byturn length coordination by a non-parametric statistical test as shown below.

Turn-by-Turn Length Coordination Test
Our null hypothesis is that there is no turn-by-turn coordination, so that all observed correlations are due to contextual factors. We now describe a procedure for testing this hypothesis.
We denote the pairwise set of exchanges in a specific dialogue c from originator o and respondent r as: where o k c ; r k c indicate the kth exchange (two utterances) by the originator o and respondent r in dialogue c, and K c represents the total number of exchanges in c. We also define the aggregated set of exchanges of user o 2 O and user r 2 R as: where C o,r represents all the dialogues that involved user o and r. We can rewrite S O R element-wise as where N = jS O R j representing number of samples. For each triplet of right hand side in Eq 8, R k is the reply utterance to O k in the dialogue C k . Finally, from S O R we obtain the set where len (Á) is a function representing the length of an utterance.
Consider now another sample, which is obtained by randomly permuting the respondent r's utterances in the set S o r,c :Ŝ where fr k c g K c k¼1 is a random permutation of r k c È É K c k¼1 . By aggregation, we have, Let us assume there is no turn-by-turn coordination, so that L O and L R are conditionally independent from each other given C. Then, it is easy to see that under this null model, the samples L(S O R ) and LðŜ O R Þ have the same likelihood, e.g., they are statistically equivalent. In other words, LðŜ O R Þ can be viewed as a new sample from the same distribution p(l o , l r ). This observation suggests the following test: We first estimate OLC from the sample L(S O R ) (denoted as OLC 0 ) and then using the within-dialogue shuffled samples LðŜ O R Þ (denoted as OLC 1 ). Under the null hypothesis, these two estimates should coincide. Conversely, if OLC 0 6 ¼ OLC 1 , then the null hypothesis is rejected, suggesting that there is turn-by-turn length coordination.
The above procedure, which we call Turn-by-Turn Length Coordination Test, is a conditional Monte Carlo test [16]. The main advantage of this non-parametric test is that it requires a smaller sample size and does not need to make particular distribution assumptions. The test is non-parametric in two ways: the permutation procedure is non-parametric as well as the estimation of mutual information. We also note that in the context of stylistic coordination, a similar test was used in Ref. [17].
The results of this test are shown in Fig 4. For the Supreme Court data, Fig 4(a) shows that both Lawyers to Judges and Judges to Lawyers have non-zero mutual information (OLC 0 ) before permutation. The Turn-by-Turn Length Coordination test shows that the mutual information decreases significantly after permutation(green confidence intervals, OLC 1 ), rejecting the null hypothesis that L O and L R are independent after conditioning on the contextual factor C. In other words, the contagion of length exists from the original utterance to the reply on a turn-by-turn level.
For the results on the Wikipedia discussion board in Fig 4(b), we are also able to reject the null hypothesis. Notice that the degree of mutual information OLC is higher for Wikipedia than for the Supreme Court. However, one cannot make a general conclusion about the exact magnitude of turn-by-turn length coordination (TLC) simply by calculating the loss, i.e., OLC 0 − OLC 1 .

Revisiting Stylistic Coordination
We demonstrated in the previous section that strong correlations in utterance length explain most of the observed stylistic coordination. However, in some situations, there are statistically significant non-zero signals even after conditioning on length, e.g., the first feature dimension Personal Pronoun in Fig 1(a) and 1(b). We now proceed to examine this remnant coordination. Specifically, we are interested in the following question: Does the non-zero conditional mutual information (after conditioning on length) represent turn-by-turn level stylistic coordination, or is it due to other contextual factors?
Toward this goal, consider the Bayesian network in Fig 5, which depicts conditional independence relations between the length variables L O and L R ; stylistic variables F m O and F m R with respect to a style feature m, and the contextual (dialogue) variable C. The solid arrow from L O to L R reflects our findings from the last section about the existence of turn-by-turn length coordination. The dashed arc between the features F m O and F m R characterizes turn-by-turn stylistic coordination. Finally, the grey arcs between C, F m O and C, F m R indicate possible contextual coordination. We use conditional mutual information to measure the Turn-by-turn Stylistic Coordination (TSC) with respect to a specific style feature m: where F m O , F m R are binary variables indicating the feature m appears or not in an utterance. Also, the Overall Stylistic Coordination(OSC) is defined as Thus, OSC is exactly the conditional mutual information introduced in Eq 2. Note that, even after conditioning on length, F m O and F m R are still dependent of each other because they are sharing the contextual factor C. [18]) Again, a direct measure of turn-by-turn stylistic coordination corresponds to non-zero TSC in Eq 12. However, TSC is hard to evaluate due to lack of sufficient samples. Furthermore, the shuffling test from the previous sections is not directly applicable here either, because it needs to be done in way that keeps the correlations between L O and L R intact: In other words, one can exchange utterances that have the same lengths. Since most dialogues are rather short, this type of shuffling test is not feasible, and one needs an alternative approach.

Turn-by-Turn Stylistic Coordination Test
Our proposed test is based on the following idea: if we can rule out the influence of the contextual factors on stylistic correlations, then any non-zero conditional mutual information can be only explained by turn-by-turn stylistic coordination, i.e., OSC = TSC. Thus, the null hypothesis is that there is contextual level coordination in stylistic features. We emphasize that by contextual coordination, we are actually referring to the links from C to F O and C to F R in Fig 5. We follow the same notation and methodology used in previous sections. By Eq 8, let us denote the mixed length and stylistic feature set of S O R as: where f m (Á) is a binary function represents whether the style feature m in an utterance appears or not.
Consider now the shuffling procedure: we randomly permute respondent's utterances within a dialogue and obtain the setŜ O R in Eq 11. We also define the length and feature set of S O R : Clearly, the permutation destroys the turn-by-turn level coordination in both length and style. Thus, any remnant correlation must be due to contextual coordination, e.g., the fork This provides a straightforward test for the existence of contextual coordination. Indeed, we simply need to estimate the overall stylistic coordination OSC 1 using the shuffled sample LF m ðŜ O R Þ. If OSC 1 is larger than zero, then there is necessarily contextual coordination. On the other hand, if OSC 1 = 0, then all the observed stylistic correlations (calculated using the original non-shuffled sample) must be due to turn-by-turn stylistic coordination.
Let us first consider the results of the above test for the Supreme Court data. From Fig 6(a) and 6(b), one can see that for all the features, the corresponding CMI OSC 1 are within the zeroinformation confidence intervals, indicating that non-zero conditional mutual information (OSC before shuffling) cannot be attributed to contextual factors. In other words, the remnant correlations that are not explained by length coordination must be due to turn-by-turn level coordination.
The situation is different for Wikipedia data. Indeed , Fig 7(a) and 7(b) show that for the stylistic features with statistically significant remnant correlations even after conditioning on length (OSC), the results of the above permutation tests are rather inconclusive. Namely, although the confidence intervals of OSC 1 do overlap with the zero information confidence intervals, one cannot state unequivocally that they are zero. In other words, one cannot rule out the null hypothesis that the remnant stylistic coordination is due to the contextual factors rather than turn-by-turn coordination.

Discussion
In conclusion, we have suggested an information theoretic framework for measuring and analyzing stylistic coordination in dialogues. We first extracted the stylistic features from the dialogue of two participants and then used Mutual Information(MI) as a theoretically motivated measure of dependence to characterize the amount of stylistic coordination between the originator and the respondent in the dialogue. Moreover, by introducing Conditional Mutual Information(CMI), which allows us to measure the correlation between two variables after conditioning on a third variable, we are able to more accurately gauge stylistic accommodation by controlling for confounding effects like length coordination.
We then used the proposed method to revisit some of the previous studies that had reported strong stylistic coordination. While the suggestion that one person's use of, e.g., prepositions will (perhaps unconsciously) lead the other to use more prepositions is fascinating, our results indicate that previous studies have vastly overstated the extent of stylistic coordination. In particular, we showed that a significant part of the observed stylistic coordination can be attributed to the confounding effect of length coordination. We find that for both Supreme Court and

Understanding Confounding Effects in Linguistic Coordination
Wikipedia data, the coordination score is greatly diminished after conditioning on length. We also find that the significant asymmetry in stylistic coordination shown in the previous study [5] is drastically weakened after conditioning on length. In fact, our results indicate that the asymmetry in length coordination can explain almost all the observed asymmetry in stylistic coordination.
Simpson's paradox provides a famous example of how correlations observed in a population can disappear or even be reversed after conditioning on sub-populations. In an informationtheoretic framework setting, a similar "paradox" can be seen in the example illustrated by Fig 3: for L O , L R and C, the mutual information I(L O : L R ) > 0, while I(L O : L R jC) = 0. If we only look at the aggregated data, averaging over all contexts, C, i.e., I(L O : L R ), there will be artificial mutual information between L O and L R . Ideally, we could calculate I(L O : L R jC) directly, however, there may not be enough samples for us to calculate the conditional mutual information for all values of C. How can we still determine whether I(L O : L R jC) is zero or not while using all the data? We thus designed non-parametric statistical tests to solve this problem in general while making full use of the available data. More importantly, because these information-theoretic quantities directly reflect constraints on graphical models, the mystery of Simpson-like paradoxes is replaced with concrete alternatives for generative stories as depicted in Figs 3 and 5.
We also observed that for some of the stylistic markers, there was diminished but still statistically significant correlations even after conditioning on length. We again designed a nonparametric statistical test for analyzing this remnant coordination more thoroughly. Our findings suggest that for the Supreme Court data, the remnant coordination cannot be fully explained by other contextual factors. Instead, we postulate that the remnant correlations in the Supreme Court data is due to turn-by-turn level coordination. For the Wikipedia data, however, our results are less conclusive, and we cannot draw any conclusion about turn-byturn stylistic coordination. Thus, caution must be taken when making general claims about the possible origin of stylistic coordination in different settings.

Understanding Confounding Effects in Linguistic Coordination
It is possible to develop alternative tests based on a more fine-grained, token-level generative models. The main idea behind such a test is to shuffle the word tokens uttered by an individual within each dialogue, which should destroy turn-by-turn coordination. Our preliminary results based on this test suggest that most of the remnant correlations are indeed due to turn-by-turn coordination. However, we emphasize that this test requires an additional assumption whose validity needs to be verified, namely that the words used by a given speaker within a conversation are independent and identically distributed (i.i.d.). Furthermore, the test assumes stationarity, i.e., that the contextual factors do not vary within the course of the dialogue. While this assumption seems reasonable in the dialogue settings considered here, it is important to note that deviations from stationarity might be yet another serious obstacle for identifying stylistic influences [19,20]. Indeed, if we relax the stationarity condition, then any observed correlation in stylistic features might be due to temporal evolution rather than direct influence. And since any permutation-based test destroys temporal ordering, it cannot differentiate between those two possibilities.
While our work focuses on linguistic style matching, we believe that the information-theoretic method proposed can be useful for studying more general types of linguistic coordination in dialogues, such as structural priming [21,22], or lexical entrainment [23,24]. Recall that according to the structural priming hypothesis, the presence of a certain linguistic structure in an utterance affects the probability of seeing the same structure later in the dialogue. This type of turn-by-turn coordination can naturally be captured by (time-shifted) mutual information between properly defined linguistic variables. Furthermore, using the permutation tests described here, it should be possible to differentiate between historical and ahistorical mechanisms of lexical entrainment [24]. Indeed, the former mechanism assumes some type of influence/coordination between the speakers that helps them to arrive at a common conceptualization. The ahistorical mechanism, on the other hand, assumes that the speaker's choice of each term is an independent event affected by the informativeness and availability of the term, and some other factors, which, in our terminology, is analogous to contextual coordination.
In a broader context, we note that sociolinguistic analysis has been used for assessing and predicting societally important outcomes such as health behaviors, suicidal intent, and emotional well-being, to name a few examples [25][26][27][28][29]. Thus, it is imperative that such predictions are based on sound theoretical and methodological principles. Here we suggest that information theory provides a powerful computational framework for testing various hypotheses, and furthermore, is flexible enough to account for various confounding variables. Recent advances in information-theoretic estimation are shifting these approaches from the theoretical realm into practical and useful techniques for data analysis. We hope that this work will contribute to the development of mathematically principled tools that enable computational social scientists to draw meaningful conclusions from socio-linguistic phenomena.