Statistical sequence models

Early research, exemplified by Factorized Personalized Markov Chains (FPMC) [8], employed Markov chains to capture linear transition patterns between sequential behaviors. While these models laid the foundation for sequence recommendation, they suffer from a fundamental flaw: an inability to model nonlinear, long-range dependencies. In the MOOC context, this limitation is particularly pronounced due to the complex multi-step prerequisite chains between courses (e.g., “Calculus Linear Algebra Machine Learning”). More critically, such models completely lack the ability to recognize periodic patterns, failing to detect regular learner behaviors such as “weekly quiz participation” or “semester-based course planning” [7]. This results in recommendations that are severely misaligned with learners’ academic rhythms.

Deep learning-based sequence models

The advent of deep learning has offered hope for overcoming the limitations of traditional statistical sequence models. Various neural architectures have driven breakthrough improvements in temporal recommendation performance through their nonlinear modeling capabilities and efficient feature extraction mechanisms. Models such as CORE [9], SINE [10], and LightSANs [11] have formed differentiated technical approaches tailored to specific scenarios, emerging as the current mainstream comparison solutions. RNN/LSTM-based models (e.g., GRU4Rec [14]) pioneered nonlinear state transition mechanisms. By dynamically adjusting hidden layer information through gating units, they effectively capture temporal dependencies in user behavior sequences, demonstrating foundational efficacy in MOOC session-level recommendations. However, sequential computation leads to inefficient processing of long sequences, and the “vanishing gradient” makes it difficult to associate logically connected course selections at a distance (e.g., the prerequisite relationship between “Advanced Mathematics” and “Machine Learning” taken six months later). CNN-based models (e.g., Caser [15]) leverage parallel computation through convolutional filtering, excelling at extracting local enrollment patterns (e.g., sequential associations like “Python Fundamentals Advanced SQL”). However, their limited receptive field prevents modeling global periodic behaviors, and they overlook course knowledge structure dependencies, resulting in insufficient recommendation coherence.

Self-attention models (such as SASRec [12] and BERT4Rec [16]) represent the current state-of-the-art, leveraging the Transformer architecture to capture global dependencies. BERT4Rec’s bidirectional self-attention and masking mechanism further unearths cross-temporal preference correlations. LightSANs [11] addresses the quadratic complexity issue of traditional self-attention by proposing a low-rank decomposition self-attention mechanism. Combined with decoupled location encoding to separate item and location relevance, it enhances efficiency while mitigating over-parameterization. SINE [10] focuses on the challenge of dispersed user interests, optimizing recommendation diversity through its "Interest Activation - intent aggregation" framework to optimize recommendation diversity: pre-constructing a large orthogonal interest concept pool while resolving the consistency issue between training and inference in traditional multi-interest models. CORE [9] addresses feature heterogeneity by mapping user session behaviors and course attributes to a unified representation space via multilayer perceptrons, introducing contrastive learning to enhance feature consistency. This effectively combines recent behaviors with course attributes in short-term session recommendations. However, none of these models transcend the temporal domain. Research by W. Song et al. [17] reaffirms that such single-temporal-domain models exhibit significant performance deficiencies when predicting non-continuous learning requests (typically periodic patterns), highlighting structural limitations.

Graph neural networks and large language models

Recent research has increasingly emphasized the role of external knowledge structures and semantic understanding techniques in recommendation systems to address the limitations of traditional behavioral sequence data. In modeling structured knowledge, G. Zhang et al. [18] employed graph convolutional networks to jointly model complex inter-course relationships and user learning styles, effectively capturing knowledge dependency characteristics in MOOC scenarios. To address data sparsity, particularly in cold-start scenarios, F. Huang et al. [19] innovatively employed large language models to generate synthetic user behavior sequences. This semantic-level data augmentation significantly improved model performance on sparse datasets. Concurrently, multimodal information fusion [20] and semi-supervised learning methods [21] have been widely applied to extract deeper user cognitive states from multi-source data such as course discussions.

While these approaches enrich feature representations by incorporating knowledge graphs and textual semantics, they fundamentally remain supplementary enhancements to core sequence modeling. Ultimately, this external information must still be processed through time-domain-based sequence recommendation frameworks (e.g., Transformers or RNNs), failing to revolutionize the underlying mechanism of sequence modeling—namely, transitioning from singular time-domain analysis to collaborative time-domain and frequency-domain modeling. Consequently, while existing methods improve recommendation effectiveness, they have not overcome the inherent limitations of time-domain modeling and remain unable to effectively capture the intrinsic periodic patterns of user behavior in MOOC scenarios.

The frequency-enhanced hybrid attention network proposed by Du et al. [13] represents a pivotal turning point in the field of frequency-aware recommendation systems. This model leverages the Fast Fourier Transform (FFT) to uncover latent periodic patterns within user behavior. However, its design and evaluation are tailored for short-term, unstructured domains such as e-commerce, failing to adequately account for the long-term, macro-scale cycles inherent in the MOOC ecosystem—such as weekly or semester-based learning cycles—and the course structures constrained by strict prerequisite knowledge requirements.

Time domain attention module

The main task of this module is to model the user’s time-series behavioral characteristics and capture the explicit and implicit dynamic preference information in the interaction sequence. Through the Self-Attention mechanism (Self-Attention), it can effectively mine the short-term interest, long-term dependency and global context information in the sequence. This is the basic module for modeling user personalized preferences in recommender systems.

• Representation of the input sequence

The interaction history of learnercan be represented as an ordered sequence of courses , where c_i is the user’s interaction behavior with course. Each course will be mapped to d-dimensional embedding vector space, as in (1).

(1)

where E is the course embedding matrix containing the embedding vector for each course in the interaction history.

• Positional Encoding

The self-attention mechanism is inherently disordered, so Positional Encoding (Positional Encoding) needs to be introduced to preserve the order information of the sequence. Positional Encoding can be calculated by the following Formula (2).

(2)

where i denotes the position in the sequence and k denotes the dimension index. Eventually, the course embedding vectors are summed with the position encoding (3).

(3)

• Self-attention mechanism computation

The self-attention mechanism generates a weighted time series representation by computing the correlation of each pair of course embeddings in the sequence. The input embedding matrix is mapped to the Query, Key, and Value vector space, as in (4).

(4)

where is the learnable weight matrix.

d_k is the dimension of the query and key vectors (usually set to d_k = d). The dot product of the query and key vectors is used to measure the correlation between courses at different locations to generate the attention weight matrix (5).

(5)

where is the attention weight matrix and A_i,j denotes the attention weight of the ith course to the jth course. is the scaling factor, which is used to mitigate the problem of excessively large dot product values.

A weighted representation of the sequence is obtained by weighting the value vector V using the attention weights.

(6)

where is the time-domain feature matrix. Each row corresponds to the time-domain contextual representation of each course in the user sequence.

To enhance the expressive power of the model, the self-attention mechanism is often extended to Multi-Head Attention. Diverse relationships of sequences are captured through multi-group queries, key and value vector computations.

(7)

Where: . h is the number of heads, is the projection matrix of the ith head. is the output matrix.

• Residual linking and normalization

To avoid gradient vanishing and to enhance the stability of the model, residual linking with layer normalization is used.

(8)

The final generated time-domain feature H_T provides a time-dynamic representation of user behavior, which is able to capture both short-term changes and long-term stability of user interests, and provides a rich feature representation for downstream tasks.

Frequency domain modeling module

The frequency domain modeling module aims at extracting periodic features from sequences of learner behaviors in order to reveal deep patterns hidden in temporal variations. For example, a learner may regularly participate in a certain type of class (e.g., a weekly programming class) during a specific period of time, and these patterns are difficult to capture directly through the time domain. By transforming the time series to the frequency domain through Fast Fourier Transform (FFT), it is possible to identify learners as significant at specific frequencies.

• Representation of the input sequence

The input data is a sequence of user interaction history behaviors , where C_i denotes the learner’s interactions with course i and each interaction is represented as a d-dimensional embedding vector. Therefore, the user’s interaction sequence can be represented as an embedding matrix , where each is an embedding vector of the course.

• Frequency domain feature extraction (Fast Fourier Transform FFT)

In order to capture the periodic features in the learner behavior data, it is first necessary to convert the time-series data to the frequency domain. Suppose the input learner behavior data is , which x_(i) represents the i moment interaction features or embedded features. Fast Fourier Transform (FFT) is applied to convert it to frequency domain.

(9)

The Fourier transform results X_f in a frequency domain component in complex form which can be expressed as in (10).

(10)

where is the magnitude information in the frequency domain and is the phase information in the frequency domain. The magnitude information A_f in the frequency domain describes the strength of the individual frequency domain into phase in the signal, while the phase information B_f indicates the time offset of the individual frequency components.

• Frequency domain magnitude information extraction

In the course recommendation task, the magnitude information is usually more recognizable than the phase information, therefore, this paper focuses on the magnitude component A_f in the frequency domain. To extract this information, the magnitude of each frequency component can be modeled. Specifically, by calculating the magnitude of each frequency component.

(11)

The amplitude vector A_f is then modeled using the amplitude vector as a frequency domain feature.

• Frequency domain attention mechanism

In order to further highlight the frequency components that are important for the recommendation task, the frequency domain attention mechanism is introduced. This mechanism helps the model to focus on important frequency components by adaptively assigning weights to different frequency components. Assuming that the frequency domain feature A_f, is a vector of length n, the frequency domain attention mechanism can be computed by the following steps.

First, the attention weights are computed by inputting the magnitude information A_f into a fully connected layer or linear mapping to generate frequency domain attention weights.

(12)

where is the weight matrix and is the mapped feature vector. Then, this feature vector is converted to a probability distribution by Softmax operation to obtain the frequency domain attention weights.

(13)

where is the attentional weight of each frequency component, indicating the importance of that frequency component.

Next, the frequency domain magnitude vectors are weighted by the frequency domain attention weights to obtain a weighted frequency domain feature representation.

(14)

Where, denotes element-by-element multiplication. H_F is the frequency domain feature weighted by frequency domain attention.

• Frequency domain feature reconstruction

Since the frequency domain features only represent the periodic part of the signal, they usually need to be converted back to the time domain for subsequent processing. The frequency domain feature H_F can be converted back to the time domain by the inverse Fourier transform (IFFT) to obtain the time domain reconstructed feature X_F.

(15)

However, not only can frequency-domain features be used directly in time-domain reconstruction, they actually provide periodicity information in their own right, which can be important in modeling user interest.

• Output of the frequency domain enhancement module

The final output of the frequency domain enhancement module is the weighted frequency domain features H_F, which provide information about the periodicity and frequency components of user behavior, complementing the time domain features. These frequency-domain features can help capture the cyclical patterns of users’ long-term behaviors and provide an additional dimension of information for subsequent time-series recommendations.

Hybrid attention mechanism

The hybrid attention mechanism is the core module of the whole model, which is responsible for jointly modeling the time-domain features H_T and the frequency-domain features H_F, and fully exploiting the complementation between them in order to generate a multidimensional integrated representation of the user’s behaviors H.

(16)

Where, , is adaptively adjusted based on the training data to ensure that the fused features are best adapted to the preference patterns of a particular user.

Recommendation prediction layer loss function and optimization

The goal of a recommender system is to generate a personalized list of recommendations, optimizing the model’s sorting performance and prediction accuracy. When optimizing for a classification task (e.g., whether to hit a course or not), the cross-entropy loss is an effective measure of the difference between the predicted distribution and the true distribution of.

(17)

Where y_u,i is the actual label and is the probability predicted by the model. The Adam optimizer is used to update the gradient of the model parameters. Adam adjusts the strategy with an adaptive learning rate to ensure the efficiency and stability of the training process. The optimization steps are as follows. Update the parameters according to the gradient calculation. A regularization term (e.g. L2 regularization) is used to mitigate the risk of over fitting and optimize the objective function.

(18)

Where, in is the regularization coefficient and is all the parameters of the model. Through the above loss function and optimization method, the FTHAN model can effectively learn the complex patterns of user behavior and achieve efficient personalized recommendation.

Experimental setup

• Data set

This study conducted experiments on the following three publicly available datasets. The statistics of data set information see as Table 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Statistics of data set information.

https://doi.org/10.1371/journal.pone.0338738.t001

MOOCCube [22]: this dataset is designed for online education recommendation tasks and contains user learning behavior data from the large-scale catechism platform Xue Tang Online. The data includes the click records, learning duration, and learning paths of the users’ learning courses, and is suitable for verifying the effectiveness of the model in educational recommendation scenarios.

MovieLens-100k (ml-100k) [23]: a classic movie recommendation dataset containing 100,000 user ratings of movies from 944 users for 1683 movies. This dataset is widely used for the validation of recommendation algorithms and is characterized by a high density of user-item interactions.

Amazon_Books [24]: user purchase and rating data extracted from Amazon’s book categories, containing rich sequences of user behaviors, which can be used to test the performance of the model in long sequence recommendation tasks.

As shown in Table 1, the publicly available datasets used in this study (e.g., MOOCCube) all exhibit highly sparse characteristics, with sparsity rates exceeding 99.57%. This extreme sparsity represents a common challenge in the field of recommendation systems, posing significant difficulties for models to effectively learn user preferences from limited interactions.

• Evaluation metrics

To comprehensively evaluate the performance of the recommender system, this study employs five widely adopted evaluation metrics: Recall@K, Precision@K, Hit@K, MRR@K, and NDCG@K. These metrics quantitatively assess the recommendation results from various dimensions including coverage, accuracy, ranking quality, and overall utility.

(1) Recall@K measures the proportion of relevant items in the test set that are successfully recommended, reflecting the system’s ability to cover user interests:

(19)

where R_u(K) represents the top-K recommended items for user u, and T_u denotes the set of true relevant items for user u in the test set.

(2) Precision@K evaluates the accuracy of recommendations by calculating the proportion of relevant items among the top-K recommendations:

(20)

(3) Hit@K serves as a coverage indicator, examining whether at least one relevant item appears in the recommendation list:

(21)

where is the indicator function that returns 1 when the condition is true and 0 otherwise.

(4) MRR@K (Mean Reciprocal Rank) assesses the ranking quality of the recommendation system by calculating the average reciprocal rank of the first relevant item:

(22)

where indicates the rank position of the first relevant item for user u in the recommendation list, and when no relevant item is present, the rank is set to K + 1.

(5) NDCG@K (Normalized Discounted Cumulative Gain) comprehensively evaluates the overall utility of recommendation lists by considering both the ranking order and relevance scores:

(23)

(24)

where rel_i represents the relevance score of the item at position i, and is the ideal DCG value for user u (calculated by sorting all items according to their true relevance in descending order).

All experiments in this study are conducted with K = 10, consistent with the common evaluation standards in the recommender systems literature.

• Baseline model

In order to validate the effectiveness of the models proposed in this study, four representative models are selected as Baseline for comparison, CORE [9] is a sequence recommendation model, which designs Representation - Consistent Encoder (RCE) to encode session embeddings as linear combinations of item embeddings within a session LightSANs [11] introduce a low-rank decomposition self-attention mechanism, which projects a user’s historical items into a small number of fixed latent interests and utilizes item-interest interactions to generate context-aware representations. SINE [10] proposes a novel sparse interest network for sequence recommendation, where the sparse interest module is able to adaptively generate a context-aware representation for each user from a large pool of concepts, and the sparse interest module is able to adaptively generate a context-aware representation for each user from a large pool of concepts. pool to adaptively infer a sparse set of concepts for each user BERT4Rec [12]. This model employs a deep bi-directional self-attention mechanism to model sequences of user behavior.

• Experimental environment

All experiments were done in a T4 GPU environment on the Google Colab platform, with a hardware configuration including an NVIDIA T4 GPU and 16GB of video memory, a software environment Python version 3.9 and the PyTorch 2.0 deep learning framework, in addition to dependent libraries such as RecBole, NumPy, Pandas, and Matplotlib.

Experimental results and analysis

• Performance comparison with baseline

On different datasets, we compared the performance with four baseline models, and the results are shown in Table 2. The comparison shows that the model based on the hybrid attention mechanism in the frequency and time domain outperforms the other baseline models in all evaluation metrics, especially in the Recall@10 and NDCG@10 metrics. This indicates that the joint consideration of frequency and time domain information can effectively improve the performance of course recommendation systems.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Model performance comparison.

https://doi.org/10.1371/journal.pone.0338738.t002

Experimental results from three datasets demonstrate that the proposed model performs exceptionally well across most metrics, exhibiting particularly significant advantages on the representative datasets MOOCCube and Amazon_Books. On the MOOCCube dataset, although the Hit@10 metric shows only a marginal 0.1% improvement (0.4534 vs. 0.4528) over the optimal baseline model LightSANs, the model achieves significantly higher values on MRR@10 and NDCG@10 key indicators reflecting ranking quality reaching 0.2018 and 0.2618, respectively, representing significant improvements of 6.4% and 4.0% over LightSANs. This demonstrates that our model matches top models in precisely identifying users Top-10 needs, but its true strength lies in prioritizing more relevant and higher quality course recommendations.

On the ml-100k dataset, the model outperformed BERT4Rec by 6.1% in Hit@10 but slightly underperformed in NDCG@10. This indicates the model effectively uncovers users’ latent interests in this scenario, though ranking precision still holds optimization potential. Notably, the model demonstrates its strongest adaptability on the larger and more complex Amazon_Books dataset, outperforming the optimal baseline across all metrics. It achieves improvements of 1.0%, 2.8%, and 1.9% in Hit@10, MRR@10, and NDCG@10, respectively.

The proposed hybrid attention model demonstrates remarkable robustness in handling extremely sparse data (99.5% sparsity in MOOCCube), achieving state-of-the-art performance across all datasets. This success stems from the synergistic dual-pathway architecture: the temporal attention module effectively captures limited local dependencies within sparse sequences, while the frequency-domain component provides a global perspective to extract noise-resistant, steady-state interest patterns. This combination proves particularly advantageous for mild cold-start scenarios where minimal interaction data exists, as the frequency-domain analysis can infer stable user preferences from limited sequences. However, the model does not address absolute cold-start scenarios with zero interactions, as it lacks mechanisms to incorporate auxiliary information like user profiles or item content. This limitation, along with further enhancing the model’s adaptability to diverse data characteristics, represents a key direction for future research.

• Ablation experiments

In order to validate the contribution of the hybrid attention mechanism in the frequency and time domains in the model, this paper conducts ablation experiments in the dataset MOOCCube, including the three cases of using only the frequency domain, only the time domain, and a combination of the time and frequency domains, and the results of the experiments are shown in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Experimental results of ablation experiments.

https://doi.org/10.1371/journal.pone.0338738.t003

The ablation experiments provide compelling evidence for the superiority of our dual-domain fusion approach. The full model, which integrates both temporal and frequency domains, achieves the best performance across all metrics (MRR@10: 0.2018, NDCG@10: 0.2618, Hit@10: 0.4534). More importantly, the performance gap between the full model and its single-domain variants quantitatively validates our core argument: domain fusion is not merely beneficial but essential for comprehensively understanding user behavior.

The most significant improvements were observed on NDCG@10 and Hit@10 metrics, where the full model outperformed single-domain approaches by approximately 10% and 5%, respectively. This specific pattern of results indicates that the key advantage of our approach lies in its enhanced ranking quality and Top-N recommendation accuracy. We attribute this superiority to the complementary roles of the two domains: the temporal domain module effectively captures local, sequence-dependent relationships in recent learning behavior, while the frequency domain component excels at identifying global, periodic interest patterns that are robust to sparse and noisy data. This synergistic combination enables the model to generate recommendations that are not only contextually relevant in the short term but also aligned with users’ stable, long-term learning habits, thereby effectively addressing the fundamental limitations of single-domain-dependent models.

• Parameter sensitivity analysis

In order to further validate the impact of key hyperparameters in the model on the recommendation performance, this paper conducts sensitivity experiments on global_ratio (frequency domain component ratio) and spatial_ratio (spatial domain to frequency domain ratio), which are conducted in in the dataset MOOCCube.

(1) Global_ratio

The parameter global_ratio represents the proportion of frequency domain components introduced in the model, which is mainly used to control the contribution of frequency domain features to the overall recommendation effect of the model. The value of global_ratio is set to [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9,1.0], and experiments are carried out on the MOOCCube dataset, with the evaluation metrics of mrr@10, hit@10 and ndcg@10, and the results are shown in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Results of mrr@10, hit@10 and ndcg@10 for different values of the global_ratio parameter.

https://doi.org/10.1371/journal.pone.0338738.g002

From the analysis of the experimental results in the above figure, mrr@10 (Mean Reciprocal Rank@10) fluctuates less under different global_ratio settings, with an optimal value of 0.2018, which occurs when global_ratio= 0.7. ndcg@10 (Normalized Discounted Cumulative Gain@10) shows an increasing and then decreasing trend with increasing global_ratio, with an optimal value of 0.2618, which occurs at global_ratio = 0.7. hit@10 (Hit Rate@10) shows a similar trend to ndcg@10 with different global_ratio settings, with an optimal value of 0.4534. The optimal value is 0.4534, which also occurs at global_ratio is 0.7.

From the experimental results, it can be seen that the appropriate setting of global_ratio is an important factor to improve the model performance. In particular, at global_ratio = 0.7, the model achieves the best performance on all metrics (mrr@10, ndcg@10 and hit@10). This suggests that a reasonable trade-off between global_ratio settings can help improve the relevance and coverage of recommendation results in recommender systems. However, when global_ratio is too large (e.g., close to 1), the recommendation performance degrades significantly because too high a weighting causes the model to ignore the contribution of other features.

(2) Spatial_ratio

The value of spatial_ratio () represents the proportion of time domain and frequency domain features, the larger is, the more the model relies on time domain features. In this paper, the values of are [0.1, 0.3, 0.5, 0.7, 0.9], and the performance of the model on the metrics mrr@10, ndcg@10, and hit@10. The value of represents the weighting of the features in the time domain and the features in the frequency domain, and the larger is, the more the model relies on the features in the time domain. The experimental results are shown in Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Results of mrr@10, ndcg@10 and hit@10 for different values of the spatial_ratio parameter.

https://doi.org/10.1371/journal.pone.0338738.g003

From the above figure, it can be seen that = 0.5 performs the best among all the metrics, with mrr@10, ndcg@10 and hit@10 reaching the peak in the dataset MOOCCube. This indicates that when the proportion of features in the time domain and frequency domain is balanced, the model can make full use of the complementary information of the two features, thus improving the recommendation effect.