Fig 1.
A simple example illustrating the goal of our work, which is to learn temporal attention between nodes by observing the dynamics of events.
The learned attention is then used to make better future predictions. Dotted edges denote attention values yet to be updated.
Fig 2.
A recursive update of node embeddingsz and temporal attention S.
An event between nodes u = 1 and v = 3 creates a temporary edge that allows the information to flow (in pink arrows) from neighbors of node u to node v. Orange edges denote updated attention values. Normalization of attention values to sum to one can affect attention of neighbors of node v. Dotted edges denote absent edges (or edges with small values).
Fig 3.
Overview of our approach relative to DyRep [14], in the context of dynamic link prediction.
During training, events ot are observed, affecting node embeddings Z. In contrast to DyRep, which updates attention weights St in a predefined hard-coded way based on associative connections At, such as CloseFriend, we assume that graph At is unknown and our latent dynamic graph (LDG) model based on NRI [15] infers St by observing how nodes communicate. We show that learned St has a close relationship to certain associative connections. Best viewed in colour.
Table 1.
Mathematical symbols used in model description. Number of relation types r = 1 in DyRep and r > 1 in the proposed LDG.
Fig 4.
Inferring an edge of our latent dynamic graph (LDG) using two passes, according to (7)–(10), assuming an event between nodes u = 1 and v = 3 has occurred.
Even though only nodes u and v have been involved in the event, to infer the edge between them, interactions with all nodes in a graph are considered.
Table 2.
Datasets statistics used in experiments.
Fig 5.
Predicting links by leveraging training data statistics without any learning (“no learn”) turned out to be a strong baseline.
We compare it to learned models with different human-specified graphs used for associations. Here, for the Social Evolution dataset the abbreviations are following: Blog: BlogLivejournalTwitter, Cf: CloseFriend, Fb: FacebookAllTaggedPhotos, Pol: PoliticalDiscussant, Soc: SocializeTwicePerWeek, Random: Random association graph.
Fig 6.
An adjacency matrix of the latent dynamic graph (LDG), St, for one of the sparse edge types generated at the end of training, compared to randomly initialized St and associative connections available in the Social Evolution dataset at the beginning (top row) and end of training (bottom row).
A quantitative comparison is presented in Table 3.
Table 3.
Results on the Social Evolution and GitHub datasets in terms of MAR and HITS@10 metrics.
We proposed models with bilinear interactions and learned temporal attention. Bolded results denote best performance for each dataset. Comparison to other baselines and ablation studies are presented in Tables 4 and 5.
Fig 7.
(a) Training curves and test MAR for baseline DyRep, bilinear DyRep, and larger baseline DyRep, with a number of trainable parameters equal to the bilinear DyRep. (b) Training curves and test MAR for the bilinear LDG with sparse prior, showing that all three components of the loss (see (13)) generally decrease or plateau, reducing test MAR.
Table 4.
Comparison of our DyRep implementation to other baselines on the Social Evolution dataset.
Table 5.
The effect of the bilinear transformation when used in different components of our LDG model on the Social Evolution dataset.
Table 6.
Edge analysis for the LDG model with a learned graph using the area under the ROC curve (AUC, %); random chance AUC = 50%.
CloseFriend is highlighted as the relationship closest to our learned graph.
Fig 8.
Final associations of the subgraphs for the Social Evolution (top row) and GitHub (bottom row) datasets compared to attention values at different time steps selected randomly.
Fig 9.
tSNE node embeddings after training (coordinates are scaled for visualization) on the Social Evolution dataset.
Lines denote associative or sampled edges. Darker points denote overlapping nodes. Red, green, and cyan nodes correspond to the three most frequently communicating pairs of nodes, respectively.