Irregular tensor

An irregular tensor is denoted as , consisting of a collection of slice matrices, where each slice represents the k-th matrix, and K is the total number of slices. Note that the row size I_k varies across slice matrices, while the column size J remains the same for all.

PARAFAC2 decomposition

PARAFAC2 decomposition [21] has been widely used for analyzing irregular tensors. Given each k-th slice matrix of an irregular tensor , PARAFAC2 decomposes each slice into three matrices: , , and V, as illustrated in Fig 1. This decomposition is formulated as , where is an matrix, is an diagonal matrix, and V is a matrix shared across all slices with R as the target rank.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. An example of PARAFAC2 decomposition.

PARAFAC2 converts each slice matrix into , , and for , where is a diagonal matrix.

https://doi.org/10.1371/journal.pone.0336100.g001

The objective function of the PARAFAC2 decomposition is formulated as follows:

where represents the Frobenius norm of matrix X, defined as , with denoting the (i,j)-th element of matrix X.

To ensure the uniqueness of the solution, several studies [22,23] reformulate with , where is a column-orthogonal matrix, and is a shared matrix across all slices.

Knowledge graph

A knowledge graph is a collection of triplets that encode facts through entities and their relationships, denoted as . Specifically, each triplet (h,r,t) captures the semantic connection between a head entity h and a tail entity t through the relationship r. For example, the triplet represents the relationship between the book Harry Potter (head entity) and the author J.K. Rowling (tail entity) through the relation Written by in the context of book knowledge. Similarly, in the context of travel, indicates that Central Park (head entity) is related to New York City (tail entity) via the relation Located in. This structured representation of a knowledge graph facilitates the efficient integration of large-scale factual information, serving as additional contextual data. A knowledge graph is expressed as a three-dimensional tensor , which takes the form of either a regular or an irregular tensor. In this paper, we represent the knowledge graph as an irregular tensor , where each is an item-specific slice matrix that forms an (entity-relation) matrix, with rows corresponding to item-related entities and columns representing relations. A more detailed explanation of this representation is provided later in this paper.

Problem definition

We define the problems of coupled tensor factorization with knowledge graph in offline and online streaming settings as follows:

Problem 1 (Coupled Tensor Factorization with Knowledge Graph).

Given.

1. (1) A temporal irregular tensor with slice matrices
2. (2) A knowledge graph tensor with slice matrices

Find. Factor matrices , , , and for , and common factor matrices , , and with R as the target rank, where each k-th slice matrix of the temporal irregular tensor is approximated by , each k-th slice matrix of the knowledge graph tensor by , and each temporal factor matrix by .

Problem 2 (Online Coupled Tensor Factorization with Knowledge Graph). Given.

1. (1) New rows of existing slice matrices in a streaming temporal irregular tensor
2. (2) A knowledge graph tensor with slice matrices
3. (3) Pre-existing factor matrices , , , , V, H, and R of accumulated irregular tensors ,

Find. Factor matrices , , , and for , and common factor matrices , , and for the entire tensor and

where; denotes the vertical concatenation of matrices, old rows of k-th existing slice matrix are approximated by , and new rows of k-th existing slice matrix are approximated by . Each temporal factor matrix is approximated by , by , and each k-th slice matrix of the knowledge graph tensor by .

Overview

KG-CTF tackles the following challenges to achieve accurate coupled tensor factorization.

1. C1. Incorporating the static features in data. While dynamic features effectively capture temporal variations, they often fail to represent time-invariant characteristics inherent in the data. How can we effectively integrate both dynamic and static information for irregular tensor data?
2. C2. Capturing the relational patterns of knowledge graphs. Knowledge graphs are structured in a triple format, where two entities are connected through a specific relationship. Effectively learning these relational dependencies is essential for leveraging static information. How can we accurately capture the relational structures of knowledge graphs?
3. C3. Accelerating ALS updates. Traditional ALS-based PARAFAC2 decomposition methods involve multiple regularization terms, often resulting in slow and unstable convergence. How can we accelerate ALS updates while optimizing complex loss functions?

To address these challenges, we propose the following key ideas (see Fig 2):

1. I1. Formulating a loss function for PARAFAC2-based coupled tensor factorization with a knowledge graph integrates temporal irregular tensors with knowledge graph tensors, effectively incorporating both dynamic and static information.
2. I2. Relational regularization enhances the learning of complex relational dependencies among factor matrices representing entities and relationships in the knowledge graph, thereby improving the representation of static information.
3. I3. Momentum-based ALS utilizes past update directions to accelerate ALS-based factor matrix updates, ensuring faster and more stable convergence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overview of KG-CTF.

(a) We introduce a method that couples a temporal tensor with a Knowledge Graph (KG) tensor by sharing the diagonal matrix S_k. This coupling allows the factor matrices to jointly learn both temporal and static features within an irregular tensor. (b) Relational regularization captures the relationship between the factor matrices derived from the KG tensor, improving their interpretability. (c) To ensure consistent entity representations across all slices, we initialize the entity factor matrix with the updated entity embeddings from . This approach maintains coherence in entity factors across slices, promoting stable learning. (d) We apply a momentum update strategy to ensure faster and stable convergence for factor matrices. At each iteration t, the momentum update refines the current ALS update step for by incorporating the update direction from , leading to a more efficient final update step for .

https://doi.org/10.1371/journal.pone.0336100.g002

Coupled tensor factorization

We formulate a loss function that effectively learns both static and dynamic information during the decomposition of irregular tensors. To capture dynamic information, we leverage the temporally irregular tensor . As described in Eq (1), each slice matrix is factorized using the PARAFAC2 decomposition with a target rank R, yielding three factor matrices: , , and . For example, in stock data analysis, represents the time factor matrix, corresponds to the stock-specific factor matrix, and V serves as the shared stock feature matrix. The loss function designed to capture this dynamic information is formulated as follows:

(1)

Next, we introduce the knowledge graph tensor as a source of static information and integrate it with the existing temporally irregular tensor . To effectively model the knowledge graph, we represent it as an irregular tensor for two key reasons. First, the number of entities associated with each item varies, inherently resulting in an irregular tensor structure. Second, the conventional entity-relation-entity representation of knowledge graphs leads to exceedingly large and sparse entity factor matrices due to the vast number of entities involved. If the entity factor matrix was constructed using the complete set of entities, the resulting tensor would be highly sparse, making an irregular tensor representation a more effective alternative. To construct this irregular tensor for knowledge graph, we define a slice matrix for each item k, where rows correspond to entities linked to k-th item and columns represent all relation types. Specifically, each matrix is derived from knowledge graph triplets (h,r,t), where the tail entity t denotes the k-th item, the head entity h is an entity associated with that item, and r is the relation type. The (i,j)-th entry of is set to 1 if the i-th entity is connected to item k via the j-th relation, and 0 otherwise. For example, in a stock-related knowledge graph, each item corresponds to a specific stock, with representing its entity–relation matrix for each stock. Triplets such as and indicate that the stock Apple is linked to entities California and IT via the relations located in and industry of, respectively. Accordingly, in for Apple, the row for California has a 1 in the located in column, and the row for IT has a 1 in the industry of column. Therefore, we define the knowledge graph tensor as , where each slice matrix (representing entity-relation interactions for an item) is factorized into three components: , , and . Here, is the entity factor matrix, is the item-specific factor matrix, and R is the relation factor matrix. In this notation, N_k denotes the number of entities linked to the k-th item, L represents the total number of relation types, and R is the target rank. For stock datasets, is decomposed into , , and R, representing entity-specific features related to each stock, stock-level latent factors, and the relation factor matrix shared across all stocks, respectively. Therefore, the loss function for the learning of static information is formulated as follows:

To ensure consistent entity representations across different slices and mitigate the inconsistencies that arise from independently decomposing each slice in PARAFAC2, we introduce a shared entity factor matrix approach. As shown in Fig 2(c), the embedding of an entity remains the same across all slices, preserving its representation throughout the decomposition process. For each slice k, the entity factor matrix is initialized using the following strategy: (1) If an entity has appeared in previous slices, its embedding is inherited from the corresponding entity factors in , (2) otherwise, its embedding is randomly initialized. This iterative approach ensures the stability of entity representations while also enhancing their interpretability across slices.

Finally, we couple the knowledge graph tensor with the temporal irregular tensor by sharing the common axis . Since is a diagonal matrix that encodes slice-specific information, sharing it allows for simultaneous learning of the dynamic characteristics from the temporal tensor and the static properties embedded in the knowledge graph tensor. Furthermore, unlike other factor matrices (e.g., , V, , or R) that are domain-specific, represents item-level latent factors. Therefore, sharing this static matrix , which is unaffected by temporal dynamics, ensures interpretability while avoiding conflicts between modalities. The overall loss function that couples these two tensors is formulated as follows:

Regularization

We add effective regularizations to the loss function.

Relational regularization.

We introduce a relational regularization term to ensure that the factor matrices obtained from the knowledge graph tensor effectively encode relational structures. Existing methods [24–26] struggle to fully capture the intricate relational dependencies present in knowledge graphs, largely because they overlook the directional nature of these relationships. To address this shortcoming, we incorporate a regularization term that explicitly models the triple structure of the knowledge graph, thereby improving the representation of directional relationships.

Based on the approach proposed in [27], we assume that each knowledge graph triplet (h,r,t) satisfies the condition , where h, r, and t correspond to the vector representations of the head entity, relation, and tail entity, respectively. Here, the norm represents the Euclidean distance, enforcing relational consistency within the knowledge graph. For instance, in a stock-related knowledge graph, if the head entity is iPhone, the relation is produced by, and the tail entity is Apple Inc., the relationship is captured by ensuring that remains close to zero.

To effectively learn these relational dependencies, we decompose each knowledge graph slice into three factor matrices. The entity factor matrix represents the head entities, the relation factor matrix captures relational properties, and the item factor matrix serves as the representation of the tail entities. To handle cases where an item is connected to multiple entities through the same relation, the regularization term is minimized using the least squares method.

For instance, if an item i₁ is linked to entities e₂ and e₃ through relation r₁, the objective is to minimize . Given that e₂ and e₃ are fixed, this can be equivalently rewritten as minimizing . Here, represents the average embedding of e₂ and e₃. Thus, to incorporate this principle, we define the relational regularization term as follows:

where is the diagonal degree matrix associated with , and the (l,l)-th element of corresponds to the number of entities connected to the l-th relation (i.e., the sum of the elements in the l-th column of ). Additionally, is a matrix filled with ones. represents the tail, meaning the embedding of the k-th item padded L times. The matrix represents the relation factor, while represents the head, computed as the average embedding of entities linked by the same relation. Here, represents the embeddings of the N_k entities connected to the k-th item. The product yields a matrix where the l-th row contains the sum of embeddings of entities linked to k-th item through relation l. By applying , we normalize this sum by the number of entities per relation, so that the l-th row of corresponds to the average embedding of entities connected to k-th item via relation l. To further illustrate how the relational regularization term operates, we provide an example in Fig 3. This figure illustrates a case where a head stock h is connected to three entities via four relation types, resulting in a slice matrix . The regularization encourages the tail embedding of stock h, repeated four times as , to be aligned with the sum of the relation embeddings and the average head entity embeddings associated with each relation, computed as . This alignment reduces inconsistency between the stock embedding and its relational context in the knowledge graph, resulting in more semantically coherent representations across domains.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Example of relational regularization for a head stock h.

Given four relation types and three entities connected to stock h, the slice matrix encodes binary connections between entities and relations. The relational regularization term aligns the tail embeddings of stock h with the sum of the relation embeddings and the average head embeddings computed over entities connected to stock h per relation.

https://doi.org/10.1371/journal.pone.0336100.g003

Uniqueness regularization.

To enhance the interpretability of the factor matrices while preserving accuracy, we employ uniqueness regularization by reformulating as , as established in previous studies [22,23], where is a column-orthogonal matrix. This approach mitigates the issue of arbitrary rotations in factor matrices, ensuring that remains unique, up to scaling and permutation. To enforce this constraint, we incorporate the following regularization terms into Eq (1):

where is a hyperparameter to control the effect of uniqueness for .

L2 regularization.

We apply L2 regularization to the factor matrices , , V, and R to mitigate overfitting and enhance numerical stability throughout the optimization process:

Loss function.

We define the following loss function , incorporating effective regularization terms:

(2)

Algorithm 1 KG-CTF with momentum-based ALS.

Input: and for

Output: , , , , for , , , and

Parameter: target rank R and momentum strength β

1: Initialize matrices for , and ,

  and R

2: repeat

3:   for do

4:    Update using Eq (3)

5:    , and

6:    Update using Eq (4)

7:    , and

8:    Update using Eq (8)

9:    Update using Eq (6)

10:   end for

11:   Update V using Eq (5)

12:   , and

13:   Update R using Eq (9)

14:   Update H using Eq (7)

15: until the maximum iteration is reached, or the error ceases

  to decrease

Momentum update procedure

We introduce a Momentum-based ALS algorithm that leverages a momentum mechanism to enhance the convergence speed of ALS when optimizing complex loss functions. By incorporating a fraction of the prior update direction into the newly computed factor matrix, the algorithm achieves more stable and faster factor updates. For the factor matrix , let denote its value at iteration t. After performing the standard ALS update to compute , the update is further refined by incorporating a momentum term:

where β is the momentum coefficient, typically ranging between 0 and 1. During the initial iterations (e.g., t < 5), β is set to zero to allow the algorithm to stabilize without the influence of previous updates. In subsequent iterations, the momentum coefficient β is assigned a positive value to integrate a portion of the previous update direction into the current update, thus accelerating convergence and enhancing optimization efficiency. By incorporating this momentum mechanism, the ALS algorithm improves its ability to process high-dimensional data, ultimately achieving faster and more stable convergence.

Algorithm 1 presents an overview of the proposed Momentum-ALS algorithm. The algorithm starts with the random initialization of the factor matrices, followed by iterative ALS updates. For each slice k, the factor matrices are initially updated using the standard ALS procedure, and a momentum term is subsequently applied to refine the final update.

We use an alternating optimization-based update procedure, where each factor matrix is updated independently while keeping the other factor matrices fixed.

Updating . We update by setting and rearranging the terms with the following lemma:

Lemma 1. When fixing all the factor matrices except for , the following update for minimizes (Eq (2)):

(3)

□

Proof: See Appendix. □

Updating . To update , we first transform for k = 1...K into whose k-th row contains the diagonal elements of (i.e., ). Then, we update by setting and rearranging the term based on the loss function (Eq (2)). We update W row by row with the following lemma:

Lemma 2. When fixing all the factor matrices except for , the following update for the k-th row of the factor matrix which corresponds to the diagonal elements of minimizes (Eq (2)):

(4)

where is a vectorization of a matrix. and * denote the Khatri-Rao product and the element-wise multiplication, respectively. is a vector filled with ones. □

Proof: See Appendix.

Updating . We update by setting and rearranging the terms with the following lemma:

Lemma 3. When fixing all the factor matrices except for , the following update for minimizes (Eq (2)):

(5)

□

Proof: See Appendix.

Updating and . We update and , which are factor matrices for the uniqueness regularization, with the following lemma:

Lemma 4. When fixing all the factor matrices except for , the following update for the matrix minimizes the loss function (Eq (2)):

(6)

where is a column-orthogonal matrix (i.e., ), and and are left and right singular vector matrices of , respectively. □

Proof: See Appendix.

Lemma 5. When fixing all the factor matrices except for , the following update for the factor matrix minimizes the loss function (Eq (2)):

(7)

□

Proof: See Appendix.

Updating . We update by setting and rearranging the terms with the following lemma:

Lemma 6. Solving the Sylvester equation (Eq (8)) with respect to minimizes the loss function (Eq (2)) when all other factor matrices are fixed:

(8)

Note that the Sylvester equation [28] has the form , where we set to solve for . □

Proof: See Appendix. □

Updating . We update by setting and rearranging the terms with the following lemma:

Lemma 7. When fixing all the factor matrices except for , the following update for the factor matrix minimizes the loss function (Eq (2)):

(9)

□

Proof: See Appendix. □

We iteratively update the factor matrices using the update procedure in Lemmas 1 to 7. Each update is the exact closed-form ALS solution of a quadratic subproblem with all other factors fixed, which ensures monotonic decrease of the objective and convergence to a stationary local minimum.

Time complexity of KG-CTF

We provide the time complexity of KG-CTF.

Theorem 1. The time complexity of KG-CTF is given by

□

Proof: See Appendix.

According to Theorem 1, KG-CTF runs in time that scales linearly with the total number of tensor entries, ensuring practicality even for very large datasets. It is also noteworthy that, in most practical settings, the KG information tables are orders of magnitude smaller than the primary tensor (). Consequently, the cubic term  +  is asymptotically dominated by the linear term in I_k, and its contribution to the overall runtime is negligible.

Overview

OKG-CTF addresses the following challenges for accurate coupled tensor factorization in an online streaming setting.

1. C1. Incorporating static features into streaming tensor data. Existing methods in online streaming settings focus only on dynamic characteristics, ignoring the time-invariant features inherent in the data. How can we efficiently employ both dynamic and static features for streaming irregular tensor data?
2. C2. Improving efficiency in an online streaming setting. Previous coupled tensor factorization methods fail to work efficiently for streaming data due to repeated computations involving old data. How can we minimize the computational cost and space cost as new tensor data arrive over time?

We address the above challenges with the following ideas (see Fig 4):

1. I1. Formulating a loss function for PARAFAC2-based streaming coupled tensor factorization using a knowledge graph enables the effective integration of streaming temporal irregular tensors with knowledge graph tensors, capturing both dynamic and static information in an online setting.
2. I2. Avoiding explicit computations on old data enables OKG-CTF to efficiently update factor matrices by dividing the terms related to old and new data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Overview of OKG-CTF.

We formulate a method to couple a streaming temporal tensor with a knowledge graph tensor by sharing the diagonal matrix S_k. This enables factor matrices to jointly learn temporal and static features within an irregular tensor in an online setting.

https://doi.org/10.1371/journal.pone.0336100.g004

Loss function for a streaming setting

We propose a loss function designed to efficiently capture both dynamic and static features from a streaming temporal irregular tensor . The loss function in Eq (2), when applied in streaming settings, suffers from inefficiencies due to redundant computations of the accumulated tensor whenever new data are given. This highlights the need for a more effective strategy tailored for a streaming setting. Therefore, we reformulate the loss function by separating the terms related to old data from those related to new data. When new data are added to each slice matrix of the temporal irregular tensor, the loss function is defined as follows:

(10)

where () is the hyperparameter controlling the effect of forgetting factor, which determines the relative importance of newly arrived data compared to older data. Specifically, we divide into and , into and , and into and . This approach ensures efficient computation by avoiding redundant operations on old data and focusing on learning from newly arrived data. The forgetting factor plays a crucial role in controlling the balance between retaining historical information and adapting to recent changes. A smaller emphasizes recent data, enabling rapid adaptation to new patterns but increasing sensitivity to short-term noise. Conversely, a larger gives greater weight to historical information, enhancing stability while slowing adaptation to distributional shifts. Thus, defines the trade-off between adaptivity and stability, highlighting the importance of selecting an appropriate balanced value in practice.

Streaming update procedure

We propose an efficient ALS update rule optimized for streaming settings, where one factor matrix is updated independently while fixing the other factor matrices. This rule eliminates the inefficiency caused by repeatedly recalculating terms involving old data whenever new data arrive, focusing on the computation of only new data.

When new data are added, the loss function is divided into two parts: one for the old data and another for the new data. During the update process, terms related to old data are loaded without any recomputation, while only the terms associated with the new data are calculated. In the subsequent step, as additional new data arrive, the previously loaded old factor matrix is combined with the computed factor matrix to form the updated old factor matrix, which is then loaded in the next step. This method effectively avoids unnecessary computational growth as data accumulate, ensuring consistent efficiency over time.

Algorithm 2 provides an overview of the proposed Momentum-ALS algorithm for OKG-CTF in a streaming setting. It starts by randomly initializing new factor matrices, followed by iterative ALS updates. For each slice k, all factor matrices are updated using standard ALS, with a momentum term applied to refine the final updates. The updated new factor matrices are then merged with the existing ones to update the old factor matrices for the next streaming data.

Algorithm 2 OKG-CTF with momentum-based ALS.

Input: a new incoming , , pre-existing factor

  matrices , , , , V, H,

  and R

Output: updated factor matrices , ,

  , , V, H, and R

Parameter: target rank R, momentum strength β

1: Initialize matrices for

2: repeat

3:   for do

4:    Update using Eq (11)

5:    , and

6:    Update using Eq (12)

7:    , and

8:    Update using Eq (8)

9:    Update using Eq (14)

10:   end for

11:   Update V using Eq (13)

12:   , and

13:   Update R using Eq (9)

14:   Update H using Eq (15)

15: until the maximum iteration is reached, or the error ceases

  to decrease

16: Update

17: Update

Updating . We update by setting and then rearranging the terms with the following lemma:

Lemma 8. When fixing all the factor matrices except for , the following update for minimizes (Eq (10)):

(11)

□

Proof: See Appendix.

Updating . To update , we first transform for k = 1...K into whose k-th row contains the diagonal elements of (i.e., ). Then, we update by setting and then rearrange the term based on the loss function (Eq (10)). We update W row by row with the following lemma:

Lemma 9. When fixing all the factor matrices except for , the following update for the k-th row of the factor matrix which corresponds to the diagonal elements of minimizes (Eq (10)):

(12)

where is a vectorization of a matrix. and * denote the Khatri-Rao product and the element-wise multiplication, respectively. is a vector filled with ones. □

Proof: See Appendix.

Updating . We update by setting and then rearranging the terms with the following lemma:

Lemma 10. When fixing all the factor matrices except for , the following update for minimizes (Eq (10)):

(13)

□

Proof: See Appendix.

Updating and . We update and , which are factor matrices for the unique regularization, with the following lemma:

Lemma 11. When fixing all the factor matrices except for , the following update for the matrix minimizes the loss function (Eq (10)) by solving Orthogonal Procrustes problem [29] due to column-orthogonality:

(14)

where is a column-orthogonal matrix (i.e., ), and and are left and right singular vector matrices of , respectively. □

Proof: See Appendix.

Lemma 12. When fixing all the factor matrices except for , the following update for the factor matrix minimizes the loss function (Eq (10)):

(15)

□

Proof: See Appendix.

Updating and . We update by setting and , and then rearranging the terms with the following lemma:

Lemma 13. The update rule for to minimize (Eq (10)) is identical to defined in Lemma 6 for (Eq (2)). □

Lemma 14. The update rule for to minimize (Eq (10)) is identical to defined in Lemma 7 for (Eq (2)). □

We alternatively update factor matrices with our update procedure in Lemmas 8 to 14. Each update is the exact closed-form ALS solution of a quadratic subproblem with all other factors fixed, which ensures monotonic decrease of the objective and convergence to a stationary local minimum.

Time complexity of OKG-CTF

We provide the time complexity of OKG-CTF for updating the factor matrices.

Theorem 2. The time complexity of OKG-CTF is given by

□

Proof: See Appendix.

Theorem 2 establishes that OKG-CTF’s time cost is linear in the time length I_k,new of the newly arrived data, with as the tensor grows. This leads to significantly reduced computational cost, especially in long streaming sequences.

Experimental settings

We provide a detailed overview of our experimental setup, including datasets, baselines, task descriptions, evaluation metrics, and hyperparameters.

Datasets. We conduct experiments on six real-world datasets, as summarized in Tables 2 and 3. Each dataset comprises a temporal irregular tensor paired with a corresponding knowledge graph tensor. These datasets cover stock markets from the United States (S&P500, NYSE, NASDAQ), South Korea, China, and Japan. For the stock datasets, each temporal irregular tensor consists of a collection of matrices, where each matrix corresponds to an individual stock. Each slice matrix follows a (date, feature) format, with features categorized into two groups: (1) six fundamental features, including opening price, closing price, highest price, lowest price, adjusted closing price, and trading volume, and (2) 83 technical indicators derived from these fundamental features using the Technical Analysis library (https://technical-analysis-library-in-python.readthedocs.io/en/latest/). For the knowledge graph datasets, we construct triple-based knowledge graphs covering all stocks in the six datasets using the ICKG [30] model. The StockKG dataset contains a total of 89,822 entities, including 14,019 stock entities and 15 distinct types of relations. These datasets are represented as irregular tensors in the format (entity, relation, stock), where each slice matrix is structured as (entity, relation). Here, entities denote the set of associated entities for each stock, and the dataset comprises 15 distinct relation types.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Description of real-world irregular tensor data.

# of nnz indicates the number of nonzeros.

https://doi.org/10.1371/journal.pone.0336100.t002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Description of real-world irregular tensor data in a streaming setting.

Update cycle denotes the number of time steps between each update.

https://doi.org/10.1371/journal.pone.0336100.t003

We preprocess the real-world datasets by applying normalization methods based on their specific characteristics. For the six stock datasets, we perform z-normalization on each j-th column of the slice matrix X. In contrast, the knowledge graph tensor, consisting of binary values (0 or 1), remains unchanged without any additional normalization.

Competitors. We compare KG-CTF against existing PARAFAC2 decomposition methods designed for irregular tensors:

• PARAFAC2-ALS: A standard PARAFAC2 decomposition approach utilizing alternating least squares. This method iteratively updates the target factor matrix while keeping all other factor matrices fixed.
• ATOM [15]: A PARAFAC2-based method tailored for handling missing values in temporal irregular tensors. ATOM introduces temporal smoothness regularization, ensuring gradual changes in temporal factor matrices over time.
• CTF-ALS: A Coupled Tensor Factorization (CTF) approach that employs ALS to jointly decompose multiple related tensors sharing a common axis. This method is designed to capture shared patterns across coupled tensors.
• TASTE [19]: A coupled matrix and tensor factorization framework that integrates both temporal and static features. It combines a non-negative PARAFAC2 model with non-negative matrix factorization to improve joint modeling capabilities.

We compare OKG-CTF with existing streaming PARAFAC2 decomposition methods in an online streaming setting:

• PARAFAC2-ALS: A baseline PARAFAC2 decomposition method to iteratively update the target factor matrix using alternating least squares.
• SPADE [31]: An efficient PARAFAC2 decomposition method designed to update new slice matrices.
• DASH [32]: An advanced PARAFAC2 decomposition method capable of efficiently handling both new slice matrices and new rows in existing slice matrices.
• CTF-ALS: A Coupled Tensor Factorization (CTF) method using ALS to jointly factorize multiple related tensors sharing a common axis, learning shared patterns across coupled tensors.

Task. To evaluate the performance of KG-CTF and OKG-CTF, we perform a missing value prediction task on temporal irregular tensor data. We randomly divide the data into training and test entries with the following ratios: , and .

Metric. We evaluate the model performance using the Root Mean Squared Error (RMSE) computed as , where Ω represents the set of test entries in the input tensor X, and denotes the reconstructed tensor obtained from the learned factor matrices. A lower RMSE score indicates better tensor decomposition accuracy.

Hyperparameters. The hyperparameters for KG-CTF and OKG-CTF include the target rank R and the regularization strengths , and , along with the momentum coefficient β. We set the hyperparameters as follows: the target rank R to 5 for the China, Japan, and Korea datasets, and to 12 for the S&P500, NYSE, and NASDAQ datasets; the regularization strengths , , , and to 10, 1, 0.01, and 0.1, respectively; and the momentum coefficient β to 0.5.

Offline performance (Q1)

We compare the performance of KG-CTF with baseline models for the task of missing value prediction at test ratios ranging from 10% to 30%. According to Table 4, KG-CTF consistently achieves lower error rates across most datasets. For instance, in the Japan Stock dataset, KG-CTF shows an error rate approximately 1.64× lower than that of PARAFAC2-ALS at 10% test ratio and about 1.52× lower at 30% test ratio. Traditional methods reveal their limitations as they focus primarily on learning dynamic features while neglecting static features. In the NASDAQ, S&P500, and Korea Stock datasets, however, the performance gaps are less pronounced since KG coverage is skewed—a few stocks are linked to many entities while most have very few—so KG signals enhance a limited subset of stocks, reducing the overall benefit of coupling. In contrast, NYSE, China, and Japan Stock datasets exhibit more balanced entity coverage, allowing KG signals to be utilized more uniformly across stocks and resulting in more consistent performance improvements. Additionally, as the test ratio increases from 10% to 30%, the reliance of competing models on dynamic features causes their performance to degrade more significantly, further widening the gap with KG-CTF.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Performance for missing value prediction.

Note that bold and underlined fonts indicate the lowest and second-lowest errors, respectively. KG-CTF outperforms the competitors across all datasets and missing value ratios.

https://doi.org/10.1371/journal.pone.0336100.t004

Online performance for newly arrived data (Q2)

We evaluate the performance of OKG-CTF with baselines in terms of local errors and efficiency.

Local Error. We evaluate OKG-CTF’s performance by analyzing local errors, measuring the mean and standard deviation across all updates. Table 5 shows that OKG-CTF consistently achieves much lower local errors than competing models. For example, in the S&P500 dataset, OKG-CTF performs up to 1.17× better than existing models. This is because OKG-CTF learns both dynamic and static features by incorporating auxiliary information with each new data arrival, resulting in more accurate factor matrices.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Performance for missing value prediction in an online setting.

Note that bold and underlined fonts indicate the lowest and second-lowest errors. OKG-CTF outperforms the competitors across all datasets and missing value ratios.

https://doi.org/10.1371/journal.pone.0336100.t005

Efficiency. We evaluate the performance of OKG-CTF compared to competing models in a streaming setting. In Fig 5, the running time for the N-th update represents the update time for the N-th newly arrived data, rather than cumulative time. Fig 5 presents the results for all datasets in a streaming setting where new rows of existing slice matrices arrive. OKG-CTF demonstrates superior performance, achieving up to 5.7 × faster speeds compared to traditional static PARAFAC2 decomposition methods and streaming PARAFAC2 decomposition methods. Notably, in the NYSE and S&P500 datasets, Fig 5(b) and Fig 5(c) show that OKG-CTF achieves faster updates compared to competing models. This shows that OKG-CTF maintains competitive speed performance, while combining static and dynamic features to learn richer information.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Running time of OKG-CTF and competitors for a new tensor on real-world datasets.

https://doi.org/10.1371/journal.pone.0336100.g005

Scalability for newly arrived data size. We evaluate the scalability of OKG-CTF with respect to the size of a new incoming tensor by measuring its running time. We measure execution times across five update cycles [20, 40, 60, 80, 100]. The length of new rows added to existing slice matrices increases linearly with the update cycle. As shown in Fig 6, the results are presented using box plots: the orange line marks the median, the box covers the upper (Q3) and lower (Q1) quartiles, and the horizontal lines represent and . Fig 6 clearly shows that, in the setting where only new rows are added to existing slice matrices, the execution time of OKG-CTF scales linearly with the update cycle.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Scalability of OKG-CTF with respect to five update cycles: [20,40,60,80,100].

The size of new rows of existing slice matrices is linearly proportional to an update cycle.

https://doi.org/10.1371/journal.pone.0336100.g006

Ablation study (Q3)

We conduct an ablation study to evaluate how relationship learning from the knowledge graph and accelerated learning via momentum updates affect prediction accuracy for both KG-CTF and OKG-CTF. Table 6 provides the global error results for KG-CTF under 20% missing-value test ratio, while Table 7 reports the local error results for OKG-CTF under 30% test ratio. For each method, we consider three reduced variants: (i) KG-CTF-R and OKG-CTF-R, which omit the relational regularization derived from the knowledge graph; (ii) KG-CTF-M and OKG-CTF-M, which exclude the momentum updates; (iii) KG-CTF-RM and OKG-CTF-RM, which remove both components and thus resemble traditional CTF-ALS. As shown in Tables 6 and 7, KG-CTF and OKG-CTF consistently show the lowest prediction errors, confirming that incorporating both relational information and momentum updates leads to superior performance. Even though KG-CTF-R and OKG-CTF-R lose some predictive accuracy by excluding knowledge graph relationships, they still outperform their respective -RM variants. Likewise, KG-CTF-M and OKG-CTF-M underscore the benefit of momentum updates, since their performance degrades relative to the full models. These results collectively demonstrate that our models effectively exploit both static relational structure and dynamic temporal information, allowing them to learn more accurate factor matrices for irregular time-evolving tensors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Ablation study of KG-CTF.

-R and -M indicate the elimination of relational regularization and no momentum update, respectively. Bold and underlined fonts indicate the lowest and second-lowest errors. Note that both the relational regularization and momentum update contribute to improving the prediction accuracy.

https://doi.org/10.1371/journal.pone.0336100.t006

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Ablation study of OKG-CTF. -R and -M indicate the elimination of relational regularization and no momentum update, respectively. Bold and underlined fonts indicate the lowest and second-lowest errors. Note that both the relational regularization and momentum update contribute to improving the prediction accuracy.

https://doi.org/10.1371/journal.pone.0336100.t007

Hyperparameter sensitivity (Q4)

We analyze the hyperparameter sensitivity of KG-CTF and OKG-CTF by measuring prediction errors across various key hyperparameters, including target rank, relational regularization, uniqueness, L2 regularization, forgetting factor, and momentum coefficient. Our experiments test four target rank values R of [3, 6, 9, 12], five relational regularization hyperparameters of [0.01, 0.1, 1, 10, 100], five uniqueness hyperparameters of [0.01, 0.1, 1, 10, 100], five L2 regularization hyperparameters of [0.01, 0.1, 1, 10, 100], five forgetting factor hyperparameters of [0.001, 0.01, 0.1, 1, 10], and five momentum coefficients β of [0.0, 0.3, 0.5, 0.7, 0.9]. We use the S&P500 (SP) and NASDAQ (ND) datasets to illustrate KG-CTF’s performance (see Fig 7), and the China Stock (CHN) and Korea Stock (KOR) datasets to examine OKG-CTF (see Fig 8).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Hyperparameter sensitivity of KG-CTF in terms of Root Mean Squared error on S&P500 and NASDAQ datasets.

Note that SP stands for S&P500 and ND stands for NASDAQ.

https://doi.org/10.1371/journal.pone.0336100.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Hyperparameter sensitivity of OKG-CTF in terms of Root Mean Squared error on China Stock and Korea Stock datasets.

Note that CHN stands for China Stock and KOR stands for Korea Stock.

https://doi.org/10.1371/journal.pone.0336100.g008

Rank. We evaluate KG-CTF’s performance on S&P500 and NASDAQ datasets across four target rank values (3, 6, 9, 12). As shown in Fig 7(a) and 7(e), the prediction errors steadily decrease as the target rank increases, reaching their lowest at the highest rank R = 12. These results show that larger rank values consistently yield lower prediction errors, confirming that a higher rank helps capture more complex patterns in the data and improves accuracy.

Relational pattern. Both KG-CTF and OKG-CTF incorporate relational regularization derived from a knowledge graph. For KG-CTF, Fig 7(b) and 7(f) illustrate the influence of learning relationships from the knowledge graph for the NASDAQ and S&P500 datasets. Specifically, for the NASDAQ dataset, value of 1 achieves lower errors compared to smaller values such as 0.1 or 0.01. In contrast, the S&P500 dataset performs better with smaller values. This difference arises because the NASDAQ dataset contains a larger number of entities, allowing the model to effectively leverage richer relational patterns. However, excessively large values of can lead to overfitting, particularly on datasets with fewer entities like S&P500, causing the model to learn noise rather than meaningful relational structures. A similar pattern appears for OKG-CTF in Fig 8(b) and 8(f). On both the China and Korea Stock datasets, moderate value of (around 1) produces optimal performance, confirming the importance of appropriately tuning relational regularization to avoid overfitting.

Uniqueness. We assess the impact of uniqueness regularization on the performance of KG-CTF and OKG-CTF. In both KG-CTF (Fig 7(c), 7(g)) and OKG-CTF (Fig 8(c), 8(g)), we observe that an excessively large degrades performance. Although moderate helps improve the interpretability of the factor matrices, excessive regularization negatively affects the overall performance.

L2. We examine how L2 regularization affects prediction performance for both KG-CTF and OKG-CTF. As shown in Fig 7(d) and 7(h) (for KG-CTF), the error rates remain largely unchanged across different L2 regularization hyperparameter values. This stability arises from the normalized input tensors, which align the scales of the factor matrices, as well as from the abundant data in the KG-CTF datasets, which support accurate factor updates even without strong regularization. In contrast, Fig 8(d), 8(h) (for OKG-CTF) reveal that larger values clearly increase prediction errors. Because OKG-CTF operates on newly arrived smaller datasets, excessive L2 regularization overly constrains the model, shrinking parameter values prematurely and preventing it from capturing subtle, local patterns. Thus, while the large datasets in KG-CTF reduce sensitivity to L2 regularization, careful tuning of is necessary for OKG-CTF to balance between preventing overfitting and preserving essential information in smaller, more localized datasets.

Forgetting factor. OKG-CTF explicitly includes a forgetting factor for time-evolving tensors. As illustrated in Fig 8(a) and 8(e) for the China and Korea stock datasets, excessively large values of negatively impact the prediction accuracy by overly emphasizing outdated data, while very small values discard useful historical information too quickly. In Korea stock dataset, a moderate value () achieves the best performance, balancing the short-term volatility and long-term trends inherent in stock data. Thus, selecting an intermediate value for is crucial, enabling the model to dynamically adapt to recent data trends while preserving sufficient historical context for stable and accurate predictions.

Momentum coefficient. We quantitatively analyze the impact of momentum-based updates on convergence speed in both KG-CTF and OKG-CTF. We measure convergence as the first iteration (epoch) at which the RMSE drops below a dataset-specific target threshold, with lower values indicating faster convergence. As shown in Fig 9, increasing the momentum coefficient β generally leads to faster convergence across the NASDAQ, NYSE, and S&P500 datasets. For example, in NASDAQ, the model with reaches the target RMSE within just 9 iterations, whereas the baseline without momentum () requires 49 iterations—indicating a 40-iteration improvement. This result highlights that the momentum mechanism becomes particularly effective in high-dimensional, large-scale datasets. However, in the S&P500 dataset, excessively large values (e.g., ) slow down convergence or cause instability, indicating that overly strong momentum may harm training stability. Fig 10 shows similar trends in OKG-CTF. Across all datasets, consistently achieves the fastest convergence; for instance, NYSE reaches the target RMSE in only 5 iterations under this setting. By contrast, higher momentum values ( or 0.9) result in slower convergence and degraded final performance. Overall, these results confirm that moderate momentum effectively accelerates convergence, whereas excessively large values can cause oscillations and suboptimal learning.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Convergence speed of KG-CTF across different momentum coefficients β, measured by the number of iterations (epochs).

Note that ND and SP denote NASDAQ and S&P500, respectively.

https://doi.org/10.1371/journal.pone.0336100.g009

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Convergence speed of OKG-CTF across different momentum coefficients β, measured by the number of iterations (epochs).

Note that ND and SP denote NASDAQ and S&P500, respectively.

https://doi.org/10.1371/journal.pone.0336100.g010

Irregular tensor decomposition

Many studies [18,33] have introduced PARAFAC2 decomposition methods for analyzing irregular tensors. Unlike the traditional PARAFAC2-based ALS algorithm [22], RD-ALS [16] and DPar2 [14] apply preprocessing steps prior to factor matrix updates. ATOM [15] further improves the handling of temporal irregular sparse tensors, particularly those with missing values. However, they primarily focus on capturing dynamic features, while static features, which are equally crucial factors, are largely neglected. Integrating static features into the PARAFAC2 model often results in significant bias toward learning dynamic features. This limitation extends to other PARAFAC2-based approaches that do not explicitly incorporate side information. To address this, KG-CTF couples the temporal irregular tensor with the knowledge graph tensor by sharing a common axis, enabling the joint modeling of dynamic and static features while preserving the inherent relational structures among factor matrices.

Streaming tensor decomposition

In online streaming settings, tensor decomposition methods [6,34,35] have been developed to efficiently update factor matrices as new data arrive. DAO-CP [9] adapts CP decomposition by detecting changes in tensor streams and selectively reusing or recomputing factor matrices. STF [7] incorporates attention-based temporal regularization to leverage past and future information for improved online learning. For irregular tensors, SPADE [31] updates factor matrices when new slices are added, while DASH [32] extends this by handling both new rows within existing slices and new slice matrices, enabling more flexible updates. However, existing methods focus on dynamic features, neglecting static features, which are equally important. A key limitation of online PARAFAC2 models is that they cannot incorporate static information in an online learning framework, as it remains unchanged over time and does not fit within their update mechanisms. While spectral regularization has been explored in neural networks and factorization models [36–38] to incorporate auxiliary structures, its use in online tensor factorization remains limited. Meanwhile, OKG-CTF effectively integrates dynamic and static features in online irregular tensor decomposition, enabling richer information learning while maintaining efficient factor matrix updates.

Coupled tensor factorization

Previous studies [39–42] have proposed Coupled Matrix-Tensor Factorization (CMTF) methods to jointly decompose tensors and matrices. HaTen2 [43] and SCouT [44] extend CMTF by implementing distributed CP-based factorization within the MAPREDUCE framework. TASTE [19] introduces a coupled irregular tensor decomposition approach for EHR data, while C3APTION [45] builds on TASTE by integrating a (non-negative) PARAFAC2 model with a (non-negative) CP model. These methods leverage complementary information to achieve data fusion, enabling the extraction of richer information through tensor decomposition. However, relying on simple matrices to represent large-scale auxiliary information has limitations, particularly in capturing intricate relationships. In contrast, KG-CTF and OKG-CTF introduce novel data fusion methods that represent knowledge graph data as irregular tensors, effectively capturing relational structures within the factor matrices. KG-CTF [39] is our preliminary work on coupled tensor factorization, designed to jointly integrate temporal irregular tensors with knowledge graph tensors. In this work, we extend KG-CTF further to effectively handle streaming irregular tensors.