Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Resolution of grammatical tense into actual time, and its application in Time Perspective study in the tweet space

  • Sabyasachi Kamila ,

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    sabysachi.pcs16@iitp.ac.in (SK); hasanuzzaman.im@gmail.com (MH)

    Affiliation Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India

  • Mohammad Hasanuzzaman ,

    Roles Conceptualization, Formal analysis, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    sabysachi.pcs16@iitp.ac.in (SK); hasanuzzaman.im@gmail.com (MH)

    Affiliation ADAPT Centre, School of Computing, Dublin City University, Dublin, Ireland

  • Asif Ekbal,

    Roles Conceptualization, Methodology, Project administration, Supervision, Visualization, Writing – review & editing

    Affiliation Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India

  • Pushpak Bhattacharyya

    Roles Conceptualization, Methodology, Project administration, Supervision, Writing – review & editing

    Affiliation Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India

Resolution of grammatical tense into actual time, and its application in Time Perspective study in the tweet space

  • Sabyasachi Kamila, 
  • Mohammad Hasanuzzaman, 
  • Asif Ekbal, 
  • Pushpak Bhattacharyya
PLOS
x

Abstract

Time Perspective (TP) is an important area of research within the ‘psychological time’ paradigm. TP, or the manner in which individuals conduct themselves as a reflection of their cogitation of the past, the present, and the future, is considered as a basic facet of human functioning. These perceptions of time have an influence on our actions, perceptions, and emotions. Assessment of TP based on human language on Twitter opens up a new avenue for research on subjective view of time at a large scale. In order to assess TP of users’ from their tweets, the foremost task is to resolve grammatical tense into the underlying temporal orientation of tweets as for many tweets the tense information, and their temporal orientations are not the same. In this article, we first resolve grammatical tense of users’ tweets to identify their underlying temporal orientation: past, present, or future. We develop a minimally supervised classification framework for temporal orientation task that enables incorporating linguistic knowledge into a deep neural network. The temporal orientation model achieves an accuracy of 78.7% when tested on a manually annotated test set. This method performs better when compared to the state-of-the-art technique. Secondly, we apply the classification model to classify the users’ tweets in either of the past, present or future categories. Tweets classified this way are then grouped for each user which gives rise to unidimensional TP. The valence (positive, negative, and neutral) is added to the temporal orientation dimension to produce the bidimensional TP. We finally investigate the association between the Twitter users’ unidimensional and bidimensional TP and their age, education and six basic emotions in a large-scale empirical manner. Our analysis shows that people tend to think more about the past as well as more positive about the future when they age. We also observe that future-negative people are less joyful, more sad, more disgusted, and more angry while past-negative people have more fear.

Introduction

In the era of data science, written languages on social media have strengthen the research on social science in an unprecedented manner. This allows the social science research to be more data-driven [1]. The social media content has been influential for analyzing different user attributes from their language use. The studies include age, gender prediction [2, 3], psychological well being [412], and a host of other behavioral, psychological and medical phenomena [1316]. However, a very few studies exist which analyze these factors using the social media users’ cognitive structure like Time Perspective (TP).

Time has been carefully studied in a multitude of ways by philosophers, sociologists, anthropologists, and psychologists. One central point of the age-old philosophical debate is whether the time is subjective or objective. The objective paradigm has seen time as a physical phenomenon- something measurable, continuous, homogeneous and universal [17, 18]. The subjective view of time perceives it as an internal, subjective phenomenon, often called ‘psychological time’, ‘lived time’ or ‘time as it is processed by the human mind’ [19]. Within the subjective notion of time, research has focused on time estimation, subjective duration of experience or time perception, time personality, time congruity, time urgency, time intensity, polychronicity and monochronicity, time structure and perceived time use [2023].

Human TP can be defined as ‘a cognitive operation that implies both an emotional reaction to time zones (such as future, present or past) and a preference for locating action in some temporal zone’ [24]. The most dominant measure of TP, the Zimbardo Time Perspective Inventory (ZTPI) modified from the Stanford Time Perspective Inventory is argued to have addressed the shortcomings of the previous scales [25]. The ZTPI scale considers TP as bidimensional constructs by taking into account both the different temporal orientations (past, present, and future) and the valence dimension. It consists of five factors: Past-Negative, Past-Positive, Present-Hedonistic, Present-Fatalistic and Future. Although ZTPI has emerged as the leading measure of TP, some theoretical and empirical studies have pointed out the drawbacks to it due to the lack of valence in the future temporal orientation [2629].

The formation of TP is believed to be heavily influenced by various factors including upbringing, socialization, culture, education, life stressors, and other situational factors that contribute to the formation of TP orientation [25, 30, 31]. It is also claimed to be a stable personality characteristic [25]. The TP is influential as it is a non-conscious process that gives orderliness to the events by associating them to different time frames. These cognitive frames are utilized to encode, accumulate and recollect the past experiences and influence the formation of the thought processes, behaviours, and other human attributes [32].

Social science and psychological studies show that TP orientation has a profound impact on the aspects of our behavior, attitudes, emotion, educational achievements, health, sleep and dreaming patterns, choice of food, romantic partner choices, sexual behavior, risk-taking, academic goal setting, interpersonal relations, organizational behavior and perceived time pressure among the other factors [25, 30, 3236]. Previous psychological studies also demonstrate the association between TP and different human attributes such as age, gender, education [3739], psychological traits [1], depression, anxiety, anger, aggression [40] and happiness [41].

Measuring TP

Traditionally, TP is assessed by the self-report questionnaires, more dominantly by ZTPI [25]. In ZTPI measure, the respondents rate the statements in different sub-scales which are used to measure different time-perspective sub-scales. The sub-scales are Past-Positive (“It gives me pleasure to think about the past.”), Past-Negative (“I think about the bad things that have happened to me in the past.”), Present-Hedonistic (“Taking risks keeps my life from becoming boring.”), Present-Fatalistic (“Often luck pays off better than hard work.”) and Future (“I complete projects on time by making steady progress,”).

Although this self-report measure is easy to administer, it highly overlaps with the self-reported measures of personality traits (e.g., future vs conscientious) [42]. The evaluation strategies like ZTPI’s Past-Negative includes different aspects of rating positive and negative experiences [25]. But maybe people’s unpleasant experiences like sadness, depression causing these relations. Thus these measures have some limitations on the actual contribution of TP to human tendencies. These self-report measures are also a very time-consuming process.

In contrast, language-based assessments can be effectively used to alternate existing TP measures using the insights from natural language processing (NLP). Unlike self-report measures where participants are driven by selective questionnaires, a majority of social media users write true positive information regarding themselves and express their true characteristics [43]. Twitter allows easy access to the vast amount of natural language text for research purpose in comparatively lesser cost. Also, there are many tools freely available for accessing tweets of million users. Twitter data is noisy (informal, ungrammatical constructions, etc.) which makes it one of the most challenging and complex text forms to process. Thus, tweets have easy accessibility, challenges to handle and potential information to study human tendencies.

In recent time, TP has been measured from the language people use in social media and correlated with different user attributes [42, 44, 45]. In [42, 44], temporal orientation classifiers are built with supervised learning technique on the manually annotated data. The authors then correlate the user-level temporal orientation with users’ age, gender, IQ, satisfaction with life, depressive symptoms, and Big-five personality factors (conscientiousness, openness, extraversion, neuroticism, and agreeableness). Another study reported in [45] measures the temporal orientation in the same manner as the previous. In this research the authors built temporal classifier by a keyword-based approach to measure the user-level temporal orientation and correlate it with the users’ income level.

In all these studies, TP has been perceived as a unidimensional construct where only the temporal orientation dimension is considered. On the contrary, we are interested in both the unidimensional and bidimensional TP from the language people use in Twitter. Earlier psychological study shows that future oriented people are usually joyful [46]. Joy is also associated with positive sentiment [47]. But it may happen that a person is future-oriented, but s/he may always think negative about the future. In that case, although the person is future-oriented, s/he is not joyful. Therefore, bidimensional TP can give us a better insight into human attributes than unidimensional TP.

In the unidimensional measure (similar to [42, 44, 45]), we only consider the different temporal orientations to yield three-factor scale: Past, Present and Future. However, it is a fact that the tense information present in a sentence does not always incline to actual time information. For example, “I need your help in the upcoming activity of our group”. Here, the tense of the verb is present but the intended temporal orientation is future. To extract the temporal orientation automatically using only the verb tense information is difficult and may require complex hand-crafted rules. It is, therefore, very crucial to design a robust method which would be able to capture the actual temporal orientation from the sentences automatically. It is to be noted that subjunctive sentences are not in the scope of our current study. In our method, we create the training data for classification by aggregating tweets based on Twitter’s hashtag information. Our model then classifies user-level tweets into either of past, present or future categories. Tweets classified in this way are then grouped for each user to get user-level unidimensional TP. The latter measure (i.e. bidimensional TP) takes into account the sentiment view for all the temporal orientations and gives rise of a nine-factor scale: Past-Positive, Past-Negative, Past-Neutral, Present-Positive, Present-Negative, Present-Neutral, Future-Positive, Future-Negative, and Future-Neutral. Finally, we measure the relationship between the users’ TP and their age, education, and six basic emotions.

Contributions

Our resolution from syntactic tense to semantic time is performed by developing a temporal classifier based on the users’ tweets collected from Twitter. We use an attention-based Bi-directional Long Short Term Memory (Bi-LSTM) network with linguistic feature embedding for the tweet temporal classification. The word-level linguistic features include words having verb Part-of-Speech (Pos) tag and words present in a temporal knowledge-base. We incorporate a hashtag-based minimally supervised method comprising of two-pass filtering to create the past-, present- and future-oriented tweets for the training of the classification network. The performance of the temporal classifier is measured on a manually annotated test set. We then use this classifier to automatically classify a large dataset created by Preoţiuc-Pietro et al. [48] consisting of ≈10 million tweets from 5,191 users of UK population mapped to their age, education and six basic emotions.

The user-level tweets with a particular temporal orientation are further subdivided into either positive, negative or neutral using a pre-trained sentiment classifier to produce bidimensional TP. Finally, we examine how TP (unidimensional and bidimensional) of the users are related to their different attributes. For this study, we consider users’ two demographic attributes (age and education) and six basic emotion categories (joy, sadness, disgust, anger, surprise, and fear) proposed by Ekman [49]. We did not use the extensive emotion categories since the relation between the TP, and the basic emotions have not been explored on a large scale in earlier studies. Please note that our main goal is not to propose a method for emotion detection. We have used the user-level attribute data as created by Preoţiuc-Pietro et al. [48] for our current study.

In summary, our main contributions are as follows:

  • We present a method to resolve grammatical tense of users’ tweets into actual time by measuring temporal orientation from tweets.
  • We propose a minimally supervised approach for temporal orientation framework that leverages a large quantity of unlabeled data and requires no hand-annotated training corpora. Performance on gold standard data shows that our proposed method outperforms the state-of-art technique;
  • We use this method to get the temporal orientation from tweets for a large number of Twitter users and then aggregate over users to obtain users’ unidimensional TP.
  • We introduce the valence dimension to assess users’ bidimensional TP.
  • We finally explore the relationship between the Twitter users’ TP and their age, education and six basic emotions on a large-scale.

Method

Our methodology is divided into two parts. The first part deals with the extraction of temporal orientation from users’ tweets and use this information to measure their TP. We then use a pre-trained sentiment classification model [50] to further classify the users’ tweets of each temporal category into positive, negative and neutral. Second part of the method is to measure the correlations between the users’ TP and their age, education and six basic emotions.

Extracting temporal orientation from tweets

We build a temporal-orientation classification model based on the attention-based Bi-directional LSTM (Bi-LSTM) network with linguistic feature embedding to classify users’ tweets into three different categories: past, present and future. We create a dataset following a minimal supervision technique by exploiting the hashtag information for training the classification network. Given the following tweet “Let me change lanes and turn left legally”, the temporal orientation classifier should predict it as an instance of future orientation.

Attention-based Bi-LSTM network with linguistic feature embedding.

Our proposed temporal classifier is a Bidirectional Long Short Term Memory (Bi-LSTM) [51] network with attention mechanism. We depict the overall architecture in Fig 1. We often fail to capture the temporal orientation of a text using just the tense information or the existing temporal keywords. For example, the tweet “Today I have a meeting at night.” is future-oriented. Here, the temporal keyword ‘Today’ has a time sense of present, whereas the tense of the verb is also present. In another example, the tweet “Working in the same unit today with different staffs was much better.” has temporal orientation as past. Here, the words which have temporal senses (i.e. working, today, was) are placed at a distance from each other. Here, the validating temporal information (‘was’ in this case) in the tweet has a long-range dependency with others. Long Short Term Memory (LSTM) [52] has been very useful to capture these kinds of long-range dependencies for the text classification tasks [53]. This motivates us to use the LSTM network.

thumbnail
Fig 1. Architecture for temporal orientation classification.

https://doi.org/10.1371/journal.pone.0211872.g001

Bidirectional LSTM network.

LSTMs are a special kind of recurrent neural network (RNN) [54] capable of learning long-term dependencies in the text by effectively handling the vanishing or exploding gradient problem. At first, a sentence is represented by an embedding layer as a sequence of word vectors w = (w1, w2, w3, …, wN), where N is the sequence length. This embedding is then given as input to the Bi-LSTM layer. In the Bi-LSTM, the output of the forward LSTM is represented as of the sequence from left to right for each word t, and the output of the backward LSTM is represented as of the sequence in the reverse direction. Finally, the Bi-LSTM outputs are merged as .

Attention network.

Attention Neural network [55] has been efficient over the LSTM network to find the important segments of a word sequence. The important segments are assigned more weights compared to the others. The merged output representation mt is given as input to the attention layer and the network produces an attention weight vector α and a hidden representation s. If M is a matrix of the merged output vectors of Bi-LSTM, then M can be represented as M = (m1, m2, m3, …, mN), where N is the sequence length.

We first feed the merged output vector mt through a non-linear layer to get pt as a hidden representation of mt as shown in Eq 1, where Ww and bw denote to the weights and bias, respectively. (1) We then measure the importance of the word as the similarity of pt with a word level context vector pw and obtain a normalized importance weight αt passing through a softmax function (Eq 2). The word level context vector pw is randomly initialized and jointly learned during the training process. It contains contextual information of the merged output layer vectors. (2) After that, we compute the sentence vector s as a weighted sum of the word annotations based on the attention weight αt (Eq 3). (3)

Word-level features.

We incorporate two word-level linguistic features into the learning process. The first feature corresponds to the verb Part-of-Speech (PoS) information present in a tweet while the second feature is used to capture the temporal keywords present in the tweet. We obtain the PoS information using the CMU tweet-tagger [56]. We capture the temporal keywords in the tweet using a temporal knowledge-base [57]. Finally, the words with verb PoS information and the detected temporal keywords in the tweet are represented in the form of 2D embedding matrices X and Y, respectively. The embedding matrix XRd×k represents a matrix of embedding dimension d (word vector dimension) and feature-length k (number of verb PoS tags in a tweet). Each row of the matrix corresponds to a word vector of those words which are detected as verbs. Similarly, the embedding matrix YRd×l represents a matrix of embedding dimension d and feature-length l (number of temporal words in a tweet) (c.f. Fig 1). Finally, we maintain an embedding matrix E by concatenating two matrices, X and Y in the lower dimensional space.

We explain our intuition behind using these two word-level features by the following two examples: i). The sentence, “I have a nice plan for the spring festival.” has a temporal sense of ‘future’ which can be determined by the temporal keyword ‘plan’, but not by the verb ‘have’. ii). The sentence, “He lost all the hopes due to his fragile mindset.” has a temporal sense of ‘past’ which can be captured by the verb ‘lost’. Here, the temporal keyword ‘hopes’ does not help. From these two examples, we can conclude that both the words with verb PoS tags and temporal keywords are important but in different ways.

To get a good feature representation out of these two word-level features, we pass the concatenated embedding matrix E through a Convolution Neural Network (CNN). We train CNN with one convolution layer followed by one max pooling layer. In the CNN model, we use 3 filters with window sizes of 5, 6 and 7.

Merged output layer.

Finally, we concatenate the CNN output f with the Bi-LSTM attention output s. The concatenated vector (c = [s;f]) is passed through a non-linear layer for the projection into the space of the targeted temporal classes by the following equation: (4) where Wi and bi are the weight matrix and bias, respectively. Finally, the class-label is predicted using a softmax function at the output layer (Eq 5). (5)

Model parameters.

For our experiments, the loss function we used is categorical cross-entropy and the optimizer we used is Root Mean Square Propagation (rmsprop). We repeat the training for 100 number of epochs with batch size set to 128. We also employ dropout for regularization [58] with a dropout rate of 0.2 to prevent over-fitting. All of these parameters are finalized by parameter tuning with the performance obtained on 10-fold cross-validation using the grid search method. The grid search method takes a list of parameters (loss function, optimizer, no of epochs, and batch size) with each parameter having different values (e.g., for the optimizer, values are adam, rmsprop, etc.) and return each parameter with the best possible value based on the accuracy on 10-fold cross-validation. The word vectors of all the words and all the words with verb PoS and the detected temporal keywords in a Tweet are of 200 dimensions. The words are searched in the pre-trained GloVe vector model [59] (trained on 2 billion tweets containing 27 billion tokens). We also validate our model on the validation set which was 10% of the training set. In each epoch, the accuracy is obtained in the validation set. Finally, we fix the model which obtained the highest accuracy on the validation set.

Measuring user-level TP

Classifying all the tweets of a user by our temporal orientation model, unidimensional TP orientation of that user is defined by the following equation: (6) where t ∈ {past, present, or future}.

The bidimensional TP orientation of a user is defined by the following equation: (7) where t ∈ {past, present, or future}), and (s ∈ {positive, negative, or neutral} per temporal category. First, we apply the temporal orientation classifier to each user’s tweets. For sentiment classification, we use a pre-trained tweet-level sentiment classification model [50] to classify the user-level tweets into positive, negative or neutral. The sentiment classifier was trained on a dataset released by SemEval-2013 shared task. The model was trained with a diverse set of features: word n-grams, character n-grams, PoS, hashtags, sentiment lexicon, emoticons, elongated words and negation. The model achieved the macro-averaged F-score of 69.02%, and ranked first in the SemEval-2013 sentiment analysis task. Finally, for each temporal category, we find the proportion of each sentiment class (i.e positive, negative or neutral) to obtain the bidimensional TP orientation.

Correlation study

We measure the correlation between the users’ TP with their age, education and different emotional attributes (joy, sadness, disgust, surprise, anger, and fear) using a linear regression [60] model. The performance is measured using a standard metric, Pearson’s correlation coefficient r between the inferred and the target values which measure the linear association between those two values.

Datasets

For experiments, we categorize the datasets into three kinds: training, test and user-level test set. Our final training set consists of 45K English tweets, whereas the test set is consisting of 741 manually annotated English tweets. The user-level test set consists of ≈10 million English tweets from 5,191 users from UK population.

Training set

To collect the data for our hashtag-based minimally supervised method, one key issue is to identify the candidate hashtags. We rely on the trending topics (i.e. hashtags) reported on Hashtags.org website for the candidate identification. The website reports the trending hashtags in three categories: trending up, trending down, and the most popular hashtags. From the trending up and trending down categories, we manually selected those hashtags which signified any temporal (past, present/ongoing or future) events. To cover more varieties of hashtags in our data collection, we have dropped the most popular hashtags category as it does not change much for many days. We use the selected hashtags as query keywords to search for the tweets on daily basis. On each day, we select the trending hashtags and we then use each selected hashtag as a query keyword to collect the tweets using Twitter’s streaming API. We collected data for two months- September and October, 2017. This collection resulted in over 280K tweets. Few example tweets with the trending topics are depicted in Table 1.

thumbnail
Table 1. Example tweets for different temporal orientation categories with trending topics.

https://doi.org/10.1371/journal.pone.0211872.t001

The collection of the training data are based on the following three hypotheses: (a) if a trending topic is of a future event then people would write mostly futuristic tweets; (b) if a trending topic is about a past incident, then people would write more about the past but they also write about the present effects of that event; (c) the tweets of the trending present event are the most critical to handle as besides writing about the present incidents, people sometimes refer to the past incidents and also give opinion about the future effects.

To deal with the pitfalls described in the hypotheses, we filter the collected tweets using a two-pass filtering method. The method is based on two assumptions, viz. (a) every meaningful sentence should contain a verb; (b) most of the past-oriented tweets contain a verb of the past tense. After two-pass filtering, our collection resulted in over 72K weakly labeled tweets: 24K Past, 29K Present, and 19K Future. We call these tweets as weakly labelled tweets since the labels are not manually annotated, instead generated based on the manually selected hashtags which are representatives of past, present, and future temporal categories.

The first assumption is well-established in the literature, whereas the second assumption is based on our observation on the tweets and validation against a tense-based classifier. In the first pass of the filtering method, we remove the tweets which do not contain any verb. We obtain this PoS tag using the CMU tweet-tagger [56]. In the second pass of the filtering method, we remove the tweets having tense as past from the tweets of present and the future events.

The CMU tweet-tagger does not provide verbs in different sub-categories. For this reason, we also obtain the PoS tag information from the Stanford PoS-tagger [61]. This provides the subcategories of verbs (i.e. VB-Verb base form, VBD-Verb past tense, VBG-Verb gerund or present participle, VBN-Verb past participle, VBP-Verb non-3rd person singular present, and VBZ-Verb 3rd person singular present). We observed that although the Stanford PoS-tagger assigned the required verb subcategories, it also introduced false positives, i.e. incorrectly tagged some non-verbs as verbs. This is the reason why we considered only those verbs (for sub-categorization) which were identified (as verbs) by the CMU tweet-tagger.

Test set

We evaluate our temporal-orientation classifier on a manually created test set. Three annotators (post-graduate level students and native English speaker) were asked to tag the 800 randomly selected English tweets in one of the four available classes, namely past, present, future and other. The annotation guidelines were as follows:

  1. Tag a tweet as past if it talks about an event which has started as well as ended or the underlying temporal connotation of the tweet refers to the past time.
  2. Tag a tweet as present if it talks about an event which has started but not ended yet or the tweet has a present temporal connotation.
  3. Tag a tweet as future if it talks about an event which is yet to happen.
  4. Tag a tweet as other in case they found it difficult to get the exact temporal tag for the tweets.

We measure the multi-rater kappa agreement [62] among the annotators. Similar to [44], we found to have a substantial agreement (kappa value of 0.82) between the annotators for the task. Finally, we select the temporal class of a tweet based on the majority voting among the annotators. The class distribution are as follows: 375 Past, 164 Present, 202 Future, and 59 Other. For our final evaluation, we remove the Other cases from the test set and thus we use 741 tweets as the test set.

User-level test set

The dataset is developed by Preoţiuc-Pietro et al. [48]. It consists of ≈10 million tweets from 5,191 users mapped to their user-level age, education and six basic emotion features: joy, sadness, disgust, surprise, anger and fear. The authors used various models such as Logistic Regression, Support Vector Regression and Gaussian Process for Regression to train with the lexical features extracted from the tweets to predict the emotion of each tweet. In addition to lexical features a set of stylistic features including emoticons, elongated words, capitalization, repeated punctuation, number of hashtags and the clause level negation were also used to build the predictive model. Finally, all the emotions per user were aggregated and the proportion of every emotion per user was calculated. Their emotion prediction model achieved an overall accuracy of 78% on a benchmark dataset.

Results

In this section, we present the results of temporal orientation classification and present the details of correlation studies.

Results of temporal orientation classification

Our temporal orientation classifier measures the orientation of each tweet in either of the past, present or future. The classifier was trained on the training set and evaluated on the manually annotated test set. We varied the training data size starting from 3K (equally distributed among three temporal classes) to 72K (overall collection after two-pass filtering) tweets and measured the accuracy over manually annotated 741 test samples. We obtain the highest accuracy of 78.7% with 45K training instances, equally distributed among the past, present and future. We define the baseline model based on the recent state-of-the-art system [45] where a temporal keyword-based weakly-supervised approach was followed to create the training set. The features were extracted using CNN, and these were used for training an SVM classifier. We present the comparative results of temporal orientation classification task in Table 2.

thumbnail
Table 2. Accuracy for past, present, future classifications using different methods measured over test data.

Results are broken down by precision (p), recall (r), and F1-measure.

https://doi.org/10.1371/journal.pone.0211872.t002

We perform feature ablation study to understand the significance of each feature. The results are reported in Table 2. Here, we find that we get the best result when we use both the features together. Results in Table 2 show that our proposed method is the most effective in correctly classifying the past class followed by the future and present. We observe low recall for the present class as many tweets which belong to present are misclassified into either of past or future classes. One reason may be the fact that the words in the tweet which represent present tense are not in the right form (Its, theres). The present tweets are misclassified into the past in those cases where mainly the existence of the tense of a verb is past, but actually the tweet has present orientation. The tweets with future orientation are mostly misclassified into the past orientation. These kinds of misclassifications are either due to the presence of past tense or the tweet is a compound sentence which has an independent clause referring to the past orientation.

Our further analysis on tense to time disambiguation shows that in only 21.6% of the manually annotated test tweets time information is resolved using only the tense information. For the remaining tweets, tense information alone is not helpful. In those remaining test tweets, our temporal orientation classifier is able to correctly disambiguate time information for 77.3% tweets. This shows that our method can resolve many instances where only tense information cannot help. Our method miss-classifies the tweets where the tweet, itself, a bit confusing and complex. For example, the tweet “Laundry day Besides the Dr Horrible song other phenomena that I enjoy involves just perusing the Juke Box-the songs just play in my head.” is present-oriented but our method classifies it as past.

Results of user-level TP

Afterwards, all the results and analysis are on the user-level test set. Fig 2 illustrates the user-level distribution of the tweets’ classes.

thumbnail
Fig 2. Kernel density estimate of user-level orientation.

(A) Kernel density estimate of user-level orientation of past, present and future; (B) Kernel density estimate of user-level orientation of past-negative, past-positive and past-neutral; (C) Kernel density estimate of user-level orientation of present-positive, present-negative and present-neutral; (D) Kernel density estimate of user-level orientation of future-positive, future-negative and future-neutral.

https://doi.org/10.1371/journal.pone.0211872.g002

The mean proportions of users’ tweets are shown in Table 3. Within users, the mean proportions of past, present, and future tweets were 0.32, 0.22 and 0.46, respectively. Among most users, the majority of messages are classified as future, while present-oriented messages are the least frequent. Within users’ past, present and future orientation, neutral messages were the most frequent while negative messages were the least frequent.

thumbnail
Table 3. The mean proportions of users’ tweets into different TP categories.

https://doi.org/10.1371/journal.pone.0211872.t003

Correlation results and discussion

We measure the predictive power of TP by performing regression on different users’ attributes. We analyze the correlation between the users’ TP and their age, education and six basic emotions. In this section, all the discussions and analyses are based on the correlation results over the user-level test set. All the correlation coefficient values are found to be statistically significant by Fisher’s R-to-Z transformation (Bonferroni corrected) with p <.001.

The correlation between the users’ age and their TP is shown in Fig 3. Evaluation results show that the users’ past orientation is positively correlated with age (r = 0.47). It signifies that people become past oriented when they age. This is in-line with a recent literature [44]. It is also to be noted that the users’ age is negatively correlated with future orientation (r = -0.46) which suggests that the users think less about the future when they age. Considering the valence dimension we obtain a correlation coefficient of r = 0.34 between the users’ future-positive orientation and their age which implies that users become more positive about the future when they age.

thumbnail
Fig 3. Correlation between the users’ TP and their age.

The Y-axis values represent the correlation coefficients. The first column represents the correlation coefficient between users’ unidimensional TP and their Age. Rest entries are for bidimensional TP vs Age (Eg. First row, second column entry represents the correlation coefficient value between past-positive and Age).

https://doi.org/10.1371/journal.pone.0211872.g003

The users’ standardized frequency of TP over their age is shown in Fig 4. We observe in Fig 4A that the trend of users’ temporal orientation changes after the age of 27 to 28. Users’ past orientation increases sharply until this period and then increases steadily. Users’ present orientation increases steadily till this period and then also decreases steadily. The future orientation of the user decreases sharply till this period and then becomes almost stable. Fig 4B shows that the users’ orientation of past-positive changes steadily with age. The past-negative orientation decreases with the increase of age while the past-neutral orientation decreases until age 30 and then increases steadily. We observe the same kind of patterns in Fig 4C and 4D where a change of trend is also seen near the age of 30.

thumbnail
Fig 4. The users’ standardized frequency of TP over their age.

Smoothing performed using loess [63] smoothing estimates. (A) The users’ standardized frequency of past, present and future orientation over their age; (B) The users’ standardized frequency of past-negative, past-positive and past-neutral over their age; (C) The users’ standardized frequency of present-positive, present-negative and present-neutral over their age; (D) The users’ standardized frequency of future-positive, future-negative and future-neutral over their age.

https://doi.org/10.1371/journal.pone.0211872.g004

We measure the correlation between the users’ TP and their education in two sub-categories: education:graduate_degree, and education: high_school. The correlation results are shown in Fig 5.

thumbnail
Fig 5. Results of correlation coefficient.

(A) Correlation between the users’ TP and Education:graduate_degree; (B) Correlation between the users’ TP and Education:high_school.

https://doi.org/10.1371/journal.pone.0211872.g005

Literature suggests that students’ temporal orientation can be useful for analyzing their academic engagement [64, 65]. In the psychological literature, education has been associated with the future orientation [64]. Our empirical results depicted in Fig 5A show that users with education as graduate_degree are future oriented (r = 0.21). The addition of valence dimension suggests that users with education graduate_degree are relatively more neutral about their future orientation (r = 0.30). Users with education high_school are seemed to be past oriented (r = 0.12) (Fig 5B). However, when we add the valence dimension we observe that these users are relatively more future-negative oriented (r = 0.20).

The first emotional attribute we consider is joy. In the psychological literature, we found that joy was related to future TP [46]. It was also associated with positive sentiment [47]. The result in Fig 6A shows that there is a positive correlation between future TP and joy (r = 0.31). We also find that joy has a negative correlation with the present and past orientation. It indicates that the future-oriented people are more joyful which is in-line with the literature. In case of bidimensional TP, we find that joyful people are more neutral about the future which is not the case found in the literature (joy vs. positive sentiment). One possible reason may be due to the miss-classifications of many positive classes into neutral by the sentiment classification model.

thumbnail
Fig 6. Correlation coefficient results.

(A) Correlation between the users’ TP and Joy; (B) Correlation between the users’ TP and Sadness.

https://doi.org/10.1371/journal.pone.0211872.g006

The second emotional attribute sadness was associated with past and present orientation [66, 67] in the psychological literature. It was also associated with negative sentiment [68]. The results in Fig 6B show that sadness has a higher correlation with future-negative followed by past-negative and present-negative which indicates that the sad people are negative minded. Moreover, they are relatively more negative about the future followed by the past and the present. In this case, considering only the unidimensional TP is somehow misleading as we see that sadness is positively correlated to the future (r = 0.11) but negatively correlated to the past and present orientation.

Our experimental results in Fig 7A show a positive correlation for disgust with the past (r = 0.41) and negative correlations with the present and future which indicates that past oriented people are more disgusted. When we consider bidimensional TP, we find relatively higher correlation between future-negative and disgust (r = 0.66) which indicates that people with disgust emotion possess more negative view of the future. In literature, disgust was related to negative sentiment [68] while the relationship between disgust and the temporal orientation was not clear.

thumbnail
Fig 7. Correlation coefficient results.

(A) Correlation between the users’ TP and Disgust; (B) Correlation between the users’ TP and Anger.

https://doi.org/10.1371/journal.pone.0211872.g007

In the psychological literature, anger was associated with the present orientation [38, 67, 69] and negative sentiment [67, 68, 70]. In some of the prior studies, it has been shown that anger and joy are similar in case of emotional arousal but different in valence (negative and positive, respectively) [71]. The study presented in [67] has shown that impact of memories of the negative and catastrophic past events can evoke negative emotion like anger. This suggests that there is also a relationship between anger and past orientation. Our experimental results in Fig 7B show that anger has a positive correlation (r = 0.43) with the past orientation and negative correlation with the present and future orientation. In case of bidimensional TP, we find that the angry people express relatively more negative sentiment towards the future (r = 0.59) than the past and the present.

The relationship between the emotional attribute surprise and the TP is not extensively studied in the literature. We find a positive correlation value (r = 0.15) between present orientation and surprise as shown in Fig 8A. It signifies that the present TP is more related to surprise. Some literature say that surprise is related to both the positive and negative sentiment based on the pleasantness and unpleasantness of the incident related to surprise [72]. We find that people while surprised think relatively more negative about the future (r = 0.18) than the present and the past.

thumbnail
Fig 8. Correlation coefficient results.

(A) Correlation between the users’ TP and Surprise; (B) Correlation between the users’ TP and Fear.

https://doi.org/10.1371/journal.pone.0211872.g008

Fear is an emotional arousal of something which does not exist at present. In the psychological literature, fear is associated with negative sentiment [68, 70] and is said to occur in response to a pending mismatch [71] or generated by an anticipated state [67, 73, 74]. The results in Fig 8B show that fear is correlated with the past TP (r = 0.42). However, when we consider bidimensional TP, we find that fear has relatively higher correlation with the past-negative followed by the present-negative and future-negative.

Although some of the findings are linked to the literature, but those research were done at the psychological or theoretical level for a small number of users. In contrast, our study conducts a large scale empirical evaluation, and finds the correlation for a large number of users based on their social media tweets.

Limitations and future scope

Here, we acknowledge the possible limitations of this study. Firstly, (i). We have not considered atemporal (where the sentence has no time sense involved) class as a separate class from the present. For example, the sentence “The color of this table is red.” has no time sense but some may argue and consider this as an instance of the present. (ii). There are some sentences like “You have this tendency to lie often.” may seem to have a present time sense but it may be categorized in a separate class from the present. (iii). There are examples like “He will think that she is pregnant.” has present or future orientation depending upon the context in which the sentence is used. As our method gives only a single tag to each sentence, it will be classified in either of present or future class depending upon the pattern learned while training. Still, our tense to time resolution method has been able to resolve 78.7% of the test instances.

Secondly, our method may suffer in terms of a user’s true identity (fake account). The demographic correlations also may subject to differ depending upon the demography itself. We leave it to the scope of future works. In the future, an alternative approach for classifying the tweets can be explored for a more accurate measure of TP. In future, we shall focus on improving the performance of temporal orientation classifier as well as sentiment classifier for this purpose.

Conclusion

In this article, we have presented a very first large-scale study to assess unidimensional and bidimensional TP from the language of social media tweets. We used tweets as these are easy to access, and provide potential information to study human cognition and tendencies. For this, extracting actual time information from tweets by tense to time disambiguation become the foremost task as for many tweets grammatical tense and syntactic time is not the same. Our temporal orientation model achieves an accuracy of 78.7% on a manually annotated test set. Moreover, our approach shows how computational techniques can be effectively used to study human cognition like TP. Whereas the previous computational large-scale studies on TP concentrated on the unidimensional measure, we focused on both the unidimensional and the bidimensional TP. We finally investigated the relationship between Twitter users’ TP and their age, education and six basic emotions. This study opens up many aspects of social and psychological science which were not possible previously on a large scale. For example, how people’s TP changes over their lifetime can be an interesting way to address the effects of TP on health and psychology.

Acknowledgments

We would like to thank Dr. Martin Crane, School of Computing, Dublin City University, Ireland and Dr. Yogesh Mani Tripathi, Department of Mathematics, Indian Institute of Technology Patna for their valuable feedback regarding statistical significance test. We would also like to thank Sayantan Mitra, PhD Scholar of AI-NLP-ML Research Group, Department of Computer Science & Engineering, Indian Institute of Technology Patna for his feedback in result visualization.

References

  1. 1. Schwartz HA, Eichstaedt JC, Kern ML, Dziurzynski L, Ramones SM, Agrawal M, et al. Personality, gender, and age in the language of social media: The open-vocabulary approach. PloS one. 2013;8(9):e73791. pmid:24086296
  2. 2. Marquardt J, Farnadi G, Vasudevan G, Moens M, Davalos S, Teredesai A, et al. Age and Gender Identification in Social Media. In: Working Notes for CLEF 2014 Conference, Sheffield, UK, September 15-18, 2014.; 2014. p. 1129–1136.
  3. 3. Sap M, Park GJ, Eichstaedt JC, Kern ML, Stillwell D, Kosinski M, et al. Developing Age and Gender Predictive Lexica over Social Media. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP. Doha, Qatar; 2014. p. 1146–1151.
  4. 4. Thelwall M. Emotion homophily in social network site messages. First Monday. 2010;15(4).
  5. 5. Dodds PS, Harris KD, Kloumann IM, Bliss CA, Danforth CM. Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PloS one. 2011;6(12):e26752. pmid:22163266
  6. 6. Chmiel A, Sienkiewicz J, Thelwall M, Paltoglou G, Buckley K, Kappas A, et al. Collective Emotions Online and Their Influence on Community Life. PloS one. 2011;6(7):1–8.
  7. 7. Choudhury MD, Counts S, Gamon M. Not All Moods Are Created Equal! Exploring Human Emotional States in Social Media. In: Proceedings of the Sixth International Conference on Weblogs and Social Media. Dublin, Ireland; 2012. p. 66–73.
  8. 8. Choudhury MD, Gamon M, Counts S. Happy, Nervous or Surprised? Classification of Human Affective States in Social Media. In: Proceedings of the Sixth International Conference on Weblogs and Social Media. Dublin, Ireland; 2012. p. 66–73.
  9. 9. Choudhury MD, Counts S, Horvitz E. Social media as a measurement tool of depression in populations. In: Web Science 2013 (co-located with ECRC). Paris, France; 2013. p. 47–56.
  10. 10. Choudhury MD, Gamon M, Counts S, Horvitz E. Predicting Depression via Social Media. In: Proceedings of the Seventh International Conference on Weblogs and Social Media. Cambridge, Massachusetts, USA; 2013. p. 128–137.
  11. 11. Choudhury MD, Counts S, Horvitz E. Predicting Postpartum changes in Emotion and Behavior via Social Media. In: 2013 ACM SIGCHI Conference on Human Factors in Computing Systems. Paris, France; 2013. p. 3267–3276.
  12. 12. Paltoglou G, Theunis M, Kappas A, Thelwall M. Predicting Emotional Responses to Long Informal Text. IEEE Trans Affective Computing. 2013;4(1):106–115.
  13. 13. Kosinski M, Stillwell D, Graepel T. Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences. 2013;110(15):5802–5805.
  14. 14. Sykora M, Robertson C, Shankardass K, Feick R, Shaughnessy K, Coates B, et al. Stresscapes: Validating Linkages between Place and Stress Expression on Social Media. In: Proceedings of the 2nd International Workshop on Mining Urban Data co-located with 32nd International Conference on Machine Learning (ICML 2015), Lille, France, July 11th, 2015.; 2015. p. 80–84.
  15. 15. Boyd RL, Pennebaker JW. Language-based personality: a new approach to personality in a digital world. Current Opinion in Behavioral Sciences. 2017;18:63–68.
  16. 16. Gruebner O, Sykora M, Lowe SR, Shankardass K, Galea S, Subramanian SV. Big data opportunities for social behavioral and mental health research. Social Science & Medicine. 2017;189:167–169.
  17. 17. Pentland WE, Harvey AS, Lawton MP, McColl MA. Time use research in the social sciences. Springer; 1999.
  18. 18. Robinson JP. The time-diary method. In: Time use research in the social sciences. Springer; 2002. p. 47–89.
  19. 19. Gorman BS, Wessman AE. Images, values, and concepts of time in psychological research. In: The personal experience of time. Springer; 1977. p. 215–263.
  20. 20. Francis-Smythe J. An exploration of time personality. The University of Manchester; 1996.
  21. 21. Kaufman CF, Lane PM, Lindquist JD. Time congruity in the organization: A proposed quality-of-life framework. Journal of Business and Psychology. 1991;6(1):79–106.
  22. 22. McGrath JE. The social psychology of time: New perspectives. Sage Publications, Inc; 1988.
  23. 23. Waller MJ, Conte JM, Gibson CB, Carpenter MA. The effect of individual perceptions of deadlines on team performance. Academy of Management Review. 2001;26(4):586–600.
  24. 24. Lennings C. Self-efficacy and temporal orientation as predictors of treatment outcome in severely dependent alcoholics. Alcoholism Treatment Quarterly. 1996;14(4):71–79.
  25. 25. Zimbardo PG, Boyd JN. Putting Time in Perspective: A Valid, Reliable Individual-Differences Metric. Journal of Personality and Social Psychology. 1999;77(6):1271–1288.
  26. 26. Boniwell I. Perspectives on time. Oxford University Press New York, NY.
  27. 27. Worrell FC, Mello ZR. The reliability and validity of Zimbardo Time Perspective Inventory scores in academically talented adolescents. Educational and Psychological Measurement. 2007;67(3):487–504.
  28. 28. Sircova A, Mitina O. Zimbardo time perspective inventory: A comparison across cultures. International Journal of Psychology. 2008;43(3):43.
  29. 29. Carelli MG, Wiberg B, Wiberg M. Development and construct validation of the Swedish Zimbardo time perspective inventory. European Journal of Psychological Assessment. 2011.
  30. 30. Boniwell I, Osin E, Alex Linley P, Ivanchenko GV. A question of balance: Time perspective and well-being in British and Russian samples. The Journal of Positive Psychology. 2010;5(1):24–40.
  31. 31. Seginer R. Adolescent future orientation: An integrated cultural and ecological perspective. Online readings in psychology and culture. 2003;6(1):5.
  32. 32. Zimbardo PG, Boyd JN. Putting time in perspective: A valid, reliable individual-differences metric. In: Time perspective theory; review, research and application. Springer; 2015. p. 17–55.
  33. 33. Keough KA, Zimbardo PG, Boyd JN. Who’s smoking, drinking, and using drugs? Time perspective as a predictor of substance use. Basic and applied social psychology. 1999;21(2):149–164.
  34. 34. Lasane TP, Jones JM. Temporal orientation and academic goal-setting: The mediating properties of a motivational self. Journal of Social Behavior and Personality. 1999;14(1):31.
  35. 35. Boyd JN, Zimbardo PG. Time Perspective, Health, and Risk Taking. Understanding behavior in the context of time. 2005; p. 85–107.
  36. 36. Stolarski M, Matthews G, Postek S, Zimbardo PG, Bitner J. How we feel is a matter of time: Relationships between time perspectives and mood. Journal of Happiness Studies. 2014;15(4):809–827.
  37. 37. Webley P, Nyhus EK. Parents’ influence on children’s future orientation and saving. Journal of Economic Psychology. 2006;27(1):140–164.
  38. 38. Drake L, Duncan E, Sutherland F, Abernethy C, Henry C. Time Perspective and Correlates of Wellbeing. Time & Society. 2008;17(1):47–61.
  39. 39. Adams J, Nettle D. Time Perspective, Personality and Smoking, Body Mass, and Physical Activity: An Empirical Study. British journal of health psychology. 2009;14(1):83–105. pmid:18435866
  40. 40. Desmyter F, De Raedt R. The relationship between time perspective and subjective well-being of older adults. Psychologica Belgica. 2012;52(1):19–38.
  41. 41. Kammann R, Flett R. Affectometer 2: A scale to measure current level of general happiness. Australian journal of psychology. 1983;35(2):259–265.
  42. 42. Park G, Schwartz HA, Sap M, Kern ML, Weingarten E, Eichstaedt JC, et al. Living in the Past, Present, and Future: Measuring Temporal Orientation With Language. Journal of personality. 2017;85(2):270–280. pmid:26710321
  43. 43. Naaman M, Boase J, Lai C. Is it really about me?: message content in social awareness streams. In: Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, CSCW 2010. Savannah, Georgia, USA; 2010. p. 189–192.
  44. 44. Schwartz HA, Park G, Sap M, Weingarten E, Eichstaedt J, Kern M, et al. Extracting Human Temporal Orientation in Facebook Language. In: Proceedings of The 2015 Conference of the North American Chapter of the Association for Computational Linguistics-Human Language Technologies (NAACL). Denver, Colorado, USA; 2015. p. 409–419.
  45. 45. Hasanuzzaman M, Kamila S, Kaur M, Saha S, Ekbal A. Temporal Orientation of Tweets for Predicting Income of Users. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. vol. 2; 2017. p. 659–665.
  46. 46. Hornik J. The role of affect in consumers’ temporal judgments. Psychology & Marketing. 1993;10(3):239–255.
  47. 47. Gao W, Li S, Lee SYM, Zhou G, Huang CR. Joint learning on sentiment and emotion classification. In: Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM; 2013. p. 1505–1508.
  48. 48. Preoţiuc-Pietro D, Volkova S, Lampos V, Bachrach Y, Aletras N. Studying user income through language, behaviour and affect in social media. PloS one. 2015;10(9):e0138717. pmid:26394145
  49. 49. Ekman P. An argument for basic emotions. Cognition and Emotion. 1992; p. 169–200.
  50. 50. Mohammad SM, Kiritchenko S, Zhu X. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. arXiv preprint arXiv:13086242. 2013.
  51. 51. Schuster M, Paliwal KK. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing. 1997;45(11):2673–2681.
  52. 52. Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation. 1997;9(8):1735–1780. pmid:9377276
  53. 53. Zhou P, Qi Z, Zheng S, Xu J, Bao H, Xu B. Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Pooling. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers; 2016. p. 3485–3495.
  54. 54. Jain LC, Medsker LR. Recurrent Neural Networks: Design and Applications. 1st ed. Boca Raton, FL, USA: CRC Press, Inc.; 1999.
  55. 55. Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv preprint arXiv:14090473. 2014.
  56. 56. Gimpel K, Schneider N, O’Connor B, Das D, Mills D, Eisenstein J, et al. Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments. In: The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Portland, Oregon, USA; 2011. p. 42–47.
  57. 57. Hasanuzzaman M, Dias G, Ferrari S, Mathet Y. Propagation Strategies for Building Temporal Ontologies. In: 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Gothenburg, Sweden; 2014. p. 6–11.
  58. 58. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research. 2014;15(1):1929–1958.
  59. 59. Pennington J, Socher R, Manning C. Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP); 2014. p. 1532–1543.
  60. 60. Neter J, Kutner MH, Nachtsheim CJ, Wasserman W. Applied linear statistical models. vol. 4. Irwin Chicago; 1996.
  61. 61. Manning CD, Surdeanu M, Bauer J, Finkel JR, Bethard S, McClosky D. The stanford corenlp natural language processing toolkit. In: ACL (System Demonstrations); 2014. p. 55–60.
  62. 62. Fleiss JL. Measuring Nominal Scale Agreement among Many Raters. Psychological Bulletin. 1971;76(5):378–382.
  63. 63. Jacoby WG. Loess:: a nonparametric, graphical tool for depicting relationships between variables. Electoral Studies. 2000;19(4):577–613.
  64. 64. Horstmanshof L, Zimitat C. Future time orientation predicts academic engagement among first-year university students. British Journal of Educational Psychology. 2007;77(3):703–718. pmid:17908382
  65. 65. Richardson M, Abraham C, Bond R. Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological bulletin. 2012;138(2):353. pmid:22352812
  66. 66. Hornik J. Time estimation and orientation mediated by transient mood. The Journal of Socio-Economics. 1992;21(3):209–227.
  67. 67. Karniol R, Ross M. The motivational impact of temporal focus: Thinking about the future and the past. Annual review of psychology. 1996;47(1):593–620. pmid:15012489
  68. 68. Liu C, Chai JW, Yu R. Negative incidental emotions augment fairness sensitivity. Scientific reports. 2016;6:24892. pmid:27101931
  69. 69. Wills TA, Sandy JM, Yaeger AM. Time Perspective and Early-onset Substance Use: A Model based on Stress–coping Theory. Psychology of addictive behaviors. 2001;15(2):118. pmid:11419227
  70. 70. Fredrickson BL. The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American psychologist. 2001;56(3):218. pmid:11315248
  71. 71. De Houwer J, Hermans D. Cognition and Emotion: Reviews of Current Research and Theories. Psychology Press; 2010.
  72. 72. Westbrook RA, Oliver RL. The dimensionality of consumption emotion patterns and consumer satisfaction. Journal of consumer research. 1991;18(1):84–91.
  73. 73. Trommsdorff G. Future orientation and socialization. International Journal of Psychology. 1983;18(1-4):381–406.
  74. 74. Strathman A, Joireman J. Understanding behavior in the context of time: Theory, research, and application. Psychology Press; 2006.