Sequential recommendation

Sequential recommendation is a subfield of recommender systems that focuses on predicting what item a user will interact with next, given their past sequence of interactions. This type of system is particularly useful in domains where the order and timing of events matter, such as music streaming, online shopping, or OTT (Over The Top) services. One common approach for building sequential recommendation systems is using Recurrent Neural Networks (RNNs), which are particularly well-suited for handling sequential data. GRU4Rec [6] is a well-known model based on RNN, which modifies classic RNNs to make them more suitable for the sequential recommendation problem. Other methods like Markov Chains [8,9] or more recent Transformer-based models [10,11] are also used depending on specific use cases and requirements. One of the most recent and state-of-the-art models, SASRec [12], is based on a self-attention mechanism that allows it to capture both short and long-term semantics of user interaction history. Sequential recommendation differs from the problem of category diversified sequential recommendation since the former ignores the diversity of categories in recommendation, which is required for improving users’ experience. In this work, we propose a category diversified sequential recommendation method outperforming previous works.

Diversified recommendation

Diversified recommendation focuses on giving a variety of items in recommendation to produce diverse results while sacrificing the minimum accuracy. Recently, diversified recommendation has been studied to enhance user experience and maximize platform revenue [13]. There are two kinds of diversified recommendations: aggregate-level and individual-level.

Category diversified sequential recommendation

Category diversified sequential recommendation ensures that the items suggested to a user span a variety of categories or genres in sequential recommendation. The key challenge with category diversified sequential recommendation lies in finding an optimal balance between accuracy and category diversity. Accuracy refers to how closely the recommended items align with a user’s known preferences. Category diversity, on the other hand, involves introducing users to new categories or genres that they might find interesting based on their past behavior and similar users’ behavior. Too much emphasis on accuracy could lead to overspecialization where users are recommended items from only one category or genre, limiting their exposure to potentially interesting content from other categories. On the flip side, too much emphasis on category diversity could result in recommending items that are too far from the user’s preferences, leading to lower satisfaction.

The state-of-the-art work on category diversified sequential recommendation, ComiRec [7], controls the category diversity using a post-processing method that reranks candidate items. Unfortunately, post-processing methods are limited to selecting items from an already chosen item pool, eventually sacrificing accuracy when trying to increase category diversity.

The problem of category diversified sequential recommendation is defined formally as follows:

Definition 1. Given a user u, an item set , a category set of items, and the user’s item interaction history until timestamp t–1, the goal of category diversified sequential recommendation is to recommend top-k items that the user u will interact on timestamp t, such that the recommendation is accurate and categories of items are diverse.

The category diversity is evaluated by the following two metrics:

• Intra-List Distance@k (ILD@k): (1) where k is the number of items recommended, c(i) is the category of item i, is the set of top-k recommended items for user u, and δ is an indicator function. ILD@k represents how many pairs of items have different categories in the recommendation list of the user. It ranges from 0 to 1, and it is 1 if all categories of items in the recommendation list are distinct.
• Coverage@k (cov@k): (2) where is the number of existing categories. cov@k represent how many different categories appear in the recommendation list of the user. It ranges from 0 to 1, and 1 means that the recommendation covers all categories.

Overview

We address the following challenges to achieve a high performance in category diversified sequential recommendation:

1. C1. Reflecting Category Information in a Backbone Model. To recommend items considering the category of items, a model needs to recognize the category information. How can we reflect category information in a backbone model?
2. C2. Navigating the Trade-off Between Category Diversity and Accuracy during Model Training. During the training process, a user embedding overfits to items within the user’s preferred categories, resulting in recommendations that lack diversity and focus too narrowly on those categories. Yet, to maintain accuracy, the embeddings must still closely align with the user’s true preferences, creating a trade-off between diversity and accuracy. How can we simultaneously increase both category diversity and accuracy during the training process of a backbone model?
3. C3. Overcoming the Constraints of Diversity Control. Training imposes inherent limitations that prevent us from freely achieving the desired level of category diversity. How can we attain a broader range of diversity with the least possible impact on accuracy after the training of a model?

The main ideas of CatDive are summarized as follows:

1. I1. Multi-embedding. We create category embeddings in addition to item embeddings, and jointly learn both embeddings to exploit category information in the backbone model.
2. I2. Preference-Confidence Based Negative Sampling. During training, we sample more negative items from the user’s preferred categories to prevent the user representation overfitting on those categories, while also prioritizing high-confidence negatives based on item popularity to maintain accuracy.
3. I3. Coverage-Prioritized Reranking. We enhance and control category diversity by prioritizing underrepresented categories in the reranking process, with minimal sacrifice to accuracy, using a tunable hyper-parameter.

Fig 2 shows the overall process of CatDive. CatDive consists of two phases, model training phase and reranking phase. In the model training phase, CatDive first enhances the backbone model by replacing the traditional item embedding with a multi-embedding approach that leverages category information. Then, CatDive trains the model with negative samples selected according to the probability formed by preference-confidence based negative sampling, improving both category diversity and accuracy. In the reranking phase, CatDive improves and controls category diversity by prioritizing underrepresented categories with minimal impact on accuracy, using a tunable hyper-parameter to adjust the extent of diversity enhancement. These ideas of CatDive are applicable to various recommendation models as shown in experiments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Overview of CatDive, which consists of training and reranking phases.

https://doi.org/10.1371/journal.pone.0341419.g002

Multi-embedding

How can we reflect the category information of items in the backbone model? In many previous models, item embeddings are randomly initialized for each item and then trained. However, this does not reflect the category information of items, and is not meaningful in obtaining category diversity.

Our idea is to create the embeddings to represent items using category information. That is, we additionally create category embeddings with the item embeddings and add them to represent items as multi-embeddings, where is the number of categories in the category set , is the number of items in the item set , and d is the embedding dimension. The category embeddings are randomly initialized for each category and then trained by the model. The multi-embedding for item j is as follows:

(3)

where is the item embedding of item j, is the category embedding of category c(j), and α is a hyper-parameter to control the impact of the category embedding.

Preference-confidence based negative sampling

How can we simultaneously increase both category diversity and accuracy during the training process of the backbone model? We propose a comprehensive negative sampling strategy that integrates two key components, category weights and confidence of the samples, which mitigate category bias and improves accuracy, respectively.

In the training phase of a recommendation model, negative sampling strategy is widely used to learn from implicit feedback. Its goal is to provide negative instances that help the model distinguish between favored and unfavored items. To this end, unobserved user–item interactions are regarded as potential negatives, and a subset of them is sampled. This process is typically used in conjunction with loss functions such as Bayesian Personalized Ranking (BPR) and Sampled Softmax (SS) loss. The BPR and SS loss are defined as follows:

(4)

where represent the predicted scores of the model for user u on item i. The set D contains tuples where i⁺ is a positive item that user u has interacted with and i⁻- is a negative item sampled from the unobserved data. N_u is the set of negative samples for user u. The primary objective of these losses is to assign higher scores to positive samples and lower scores to negative samples. Consequently, items selected as negative samples are less likely to be recommended by the model.

A naive approach to negative sampling involves randomly selecting items that the user has not interacted with. However, random sampling leads to selecting negative samples primarily from items in less preferred categories of each user, which results in the model recommending only items from the user’s preferred categories, thus reducing diversity. Additionally, when negative samples are drawn from lesser-known items that the user might not have encountered, it causes the model to incorrectly assume the user dislikes those items, thereby reducing the overall accuracy of the recommendations.

To address the challenge of random negative sampling, we propose an effective approach that evaluates the suitability of each item as a negative sample using a scoring function s_u,i and select negative samples according to the score. Thus, the probability of selecting an item as a negative sample for user u based on the score s_u,i as follows:

(5)

where is the set of all potential negative samples for user u. This probability distribution ensures that items with higher scores are more likely to be selected as negative samples. In practice, when negative samples are pre-generated before training, the sampling process introduces no additional time overhead during the training phase.

We next define the scoring function s_u,i that quantifies the suitability of each item as a negative sample. The scoring function s_u,i integrates category-preference and confidence score into a unified scoring function. It is defined as:

(6)

where is the category-preference score of item i for user u and is the confidence score of item i. β is a hyper-parameter to control the level of adjustment; a larger β gives more weight to . Although β is global in our current setup, the framework is extensible to per-user , which adjusts the confidence according to user-specific preferences and features. Category-preference score ensures the category-diversity of recommendation while confidence score improves the accuracy of recommendation. The specific definitions of the category-preference score and confidence score are discussed in detail in the following subsections.

Category-preference score.

How can we prevent the model from recommending the items only in preferred categories? Our idea is to sample negative items proportionally to each user’s category preference. Through this, we avoid the overfitting of the user embedding to the preferred categories’ items. Randomly sampling from them results in more skewed recommendation towards preferred category, leading to low diversity. Instead, by selecting more items in preferred category as negative samples, the resulting recommendation becomes not skewed to the preferred category.

To achieve this, we design the category-preference score and select negative samples that align with the user’s category preference. Based on this objective, we define the following desired property for score :

Property 1. if , where int(u,x) is the number of interaction between user u and x which is either an item or a category.

Property 1 states that a preferred category should be given a higher priority for negative samples than a less preferred category. We define the score as follows:

(7)

where is the set of items that user u has interacted with. It is straightforward to show that Eq (7) satisfies the Property 1, which is summarized in Lemma 1.

Lemma 1. Eq (7) satisfies Property 1.

is added in the final negative sampling score to select negative samples for user u, thus an item with a higher is more likely to be selected as a negative sample (see Eq (6)).

Confidence score.

How can we maintain the accuracy of recommendation while improving the category diversity? Note that in the preferred categories of a user, there are possibly positive items as well as negative items. Thus, we have to carefully select negative items from them by finding high-confidence negative items that are likely to be negative items.

Our main idea to select high-confidence negative items is to consider the popularity of items. Recall that negative items should be selected among items that the user has not interacted with. Assume that a user u has not interacted with an item i. There are two possible reasons for this. First, u may not know the existence of i. Second, u does not like i even though u knows i. Then, it makes sense to select a negative sample that corresponds to the second case (u does not like i even though u knows i), since it is not clear whether u dislikes i in the first case.

Based on this idea, we design the score , which ensures that items with higher popularity are more likely to be selected as negative samples. The score should satisfy the following property:

Property 2. if , where is the set of users.

Property 2 states that a popular item should be given a higher priority for negative samples. We show in the ablation study that giving such higher priority to a popular item is effective for finding high-confidence negative items and boosting accuracy. We define the scoring function as follows:

(8)

where is the set of all items. It is clear that Eq (8) satisfies Property 2, which is summarized in Lemma 2. We denote and as and for brevity, respectively.

Lemma 2. Eq (8) satisfies Property 2.

Coverage-prioritized reranking

How can we attain a broader range of diversity with the least possible impact on accuracy after the training of the model? Existing reranking algorithms [7,19] apply penalties to items from the same category and select items in a greedy manner. However, since similar items tend to have close ranks, items from the same category often have similar scores. As a result, such minor reordering fails to include lower-ranked items from certain categories, leading to those categories being underrepresented in the final recommendation list. Instead, our approach separates coverage from scoring via a two-step procedure: (i) a coverage-first step that explicitly secures items from underrepresented categories in the candidate pool, followed by (ii) a category-aware fill that completes the list with adjusted scores conditioned on the running category histogram. This design goes beyond penalty-only reranking and widens the attainable diversity range while preserving accuracy. Algorithm 1 shows the overall process of coverage-prioritized reranking.

Algorithm 1 Coverage-prioritized reranking.

Input: recommendation score of user u for each item i in the candidate item set , category c(i) for each item i in the item set , the number k of recommendation, the number of candidate items, and hyper-parameters controlling the degree of reranking

Output: top-k recommendation list for user u

1:

2:

3: such that c_j is a unique category in and sorted in ascending order of frequency:

  // select one item from each of the least popular categories

4: for each category do

5:   select an item i^* such that

6:  

7:  

8: end for

  // iteratively fill the recommendation list considering the penalized score of candidate items

9: while do

10:   for select the item with the highest adjusted score considering category:

11:  

12:  

13: end while

14: sort( by in descending order)

15: return the final recommendation list

First, we select one item from each of the least popular categories to ensure they are represented and to enhance category coverage. This approach is necessary because items from least popular categories are unlikely to be included in the top-k recommendations if we consider only preference scores or apply minimal reranking because their preference scores tend to be lower than those of items in popular categories. In line 3, we identify the unique categories within the candidate items and then sort the categories in ascending order of their frequency. From line 4 to 8, we repeatedly select the highest-scoring item from one of the least popular categories until the desired number of items has been selected. Specifically, given a set of categories such that c_j is a unique category in and sorted in ascending order of frequency in , an item i^* with the highest prediction score from each category c is selected:

(9)

The selected item is added to the final recommendation list . The number of items to be selected in this phase is , where is a hyper-parameter to adjust the level of coverage. Higher the , the more focus on category coverage is given.

Next, from line 9 to 13 we iteratively fill the recommendation list with top-ranked items until reaching the desired number k. During each iteration, we penalize the scores of candidate items based on the category frequency of the items already present in . In line 10, we apply a penalty to the scores of the candidate items for reranking based on category frequency of items already added to :

(10)

where is a hyper-parameter that controls the degree of penalty. A high assigns stronger penalties to already represented categories, thereby increasing the overall diversity. Both and are extendable to user-specific forms, allowing personalized control over diversity within the reranking phase. In line 11, we add the highest-scoring item to the recommendation list, and in line 12, we remove it from the candidate set. This process is repeated until the recommendation list is fully populated.

Finally, in line 14, we reorder the items based on their original predicted scores before returning the final recommendation list. While diversity is not dependent on the order of the recommended items, accuracy metrics (e.g., nDCG) are, thus reordering according to the original predicted scores is crucial for maintaining a high level of accuracy.

This reranking operates on a small candidate set (top- items) and scales linearly with , adding negligible latency to the overall pipeline. Hence, category-prioritized reranking is easily deployed in large-scale or high-throughput recommendation environments.

Experimental setup

We introduce our experimental setup including datasets, evaluation protocol, baseline approaches, and evaluation metrics.

Datasets.

We use three real-world rating datasets as summarized in Table 2. All datasets are publicly released by their providers, and we use them under the terms of their respective licenses. Amazon Books and Amazon Kindle are constructed from Amazon, an online store. We use the first subcategory of items as the category. GoodReads Romance is constructed from GoodReads, a book review site. The subcategories of Romance category are used as the category.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of datasets.

https://doi.org/10.1371/journal.pone.0341419.t002

Performance

We compare the performance of CatDive and competitors on three real-world datasets in Fig 3. For the diversified methods with reranking (MMR, ComiRec, and the proposed CatDive), we adjust two hyper-parameters , for coverage-prioritized reranking in increments of 0.1, ranging from 0 to 1, and mark all points. When , equal to 0, reranking is not applied and diversity is at the lowest point. The category weight α is set to 1.0, 0.5, and 0.9 for the best performance, while the confidence weight β is set to 0.6, 0.4, and 0.8. The best point with the highest accuracy and the highest diversity in each plot is marked as a black star. Note that CatDive achieves the highest category diversity with least accuracy drop compared to others baselines considering both ILD and coverage. CatDive shows up to 152.49% higher category diversity in the similar level of accuracy and up to 39.73% higher accuracy in the similar level of category diversity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Performance of CatDive and competitors.

CatDive is the closest to the best point with the highest category diversity and accuracy among all datasets.

https://doi.org/10.1371/journal.pone.0341419.g003

Generality

To demonstrate the general applicability of CatDive, we apply it to various backbone models and present the performance comparison in Fig 4. We use GRU4Rec, SASRec, and the multi-interest module of ComiRec as the backbone models. GRU4Rec and SASRec are single-interest sequential recommendation models, whereas ComiRec operates as a multi-interest model. For GRU4Rec, α is set to 1, while β is adjusted to 0.1, 0.4, and 0.3 for Books, Kindle, and GoodReads datasets, respectively. For ComiRec, are set to , , and for the same datasets. Note that CatDive consistently enhances both category diversity and accuracy across all backbone models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Performance of CatDive applied to different backbone models.

CatDive improves both category diversity and accuracy when applied to all models.

https://doi.org/10.1371/journal.pone.0341419.g004

Ablation study

We analyze the effect of multi-embedding and preference-confidence based negative sampling in the training phase. We have three observations from the result in Table 3. First, multi-embedding significantly improves the accuracy of recommendation showing up to 31.35% improvement. Second, preference-confidence based negative sampling dramatically increases category diversity without sacrifice of accuracy, showing up to 47.89% increase with up to 7.06% increase in accuracy. Third, applying both multi-embedding and preference-confidence based negative sampling successfully balances category diversity and accuracy showing up to 36.49% increase in category diversity with up to 32.09% improvement in accuracy.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Ablation study of multi-embedding (ME) and preference-confidence based negative sampling (NS).

‘Improv.’ next to each component indicates the improvement relative to the original backbone model. The best is marked bold and the second best is underlined.

https://doi.org/10.1371/journal.pone.0341419.t003

Case study

To verify the category diversity of CatDive, we compare the recommendation results created by CatDive and other model in real-world datasets. Table 4 shows the result from Amazon Books dataset by CatDive and vanilla SASRec. We have two observations. First, given the same user’s history, CatDive recommends books of diverse categories while vanilla SASRec recommends all books from a single category that appears frequently in the user’s history. For instance, vanilla SASRec recommends books only from Romance for user 1 while CatDive recommends books from Science Fiction & Fantasy, Literature & Fiction, Humor, and Romance. Second, CatDive finds the ground-truth items that are not in the dominant category for each user. For example, the ground-truth item for user 1 is ‘Blood Fury’ in Science Fiction & Fantasy category which is not the dominant category of the user 1. This shows that considering category diversity by CatDive helps correctly selecting the target item, preventing the popularity bias of the existing model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Recommendation by CatDive and SASrec in Amazon books.

CatDive recommends books of diverse categories while SASRec recommends only a single category. SF & Fan and Lit & Fic stand for Science Fiction & Fantasy and Literature & Fiction, respectively. The ground-truth items are marked in bold.

https://doi.org/10.1371/journal.pone.0341419.t004