2.1.1 Neural CF without neighbors’ concept.

A few years ago, utilizing MF through a NN was widely used in the RS model, such as in the representation step, which initializes user and item vectors in the prediction step to compute the rating prediction and to decrease the dimension of the matrix in the system.

To represent the user and item as a vector, many current studies use a MF NN. First, the user and item vectors are initialized. Then, the MF concept is used to compute predicted rating () for learning and adjusting the user and item vectors: (2) where ⊙ denotes the element-wise product between two vectors. p_u and q_i are the user profile and the item characteristic vector, respectively. The target user’s actual rating values on the target item are labeled for the backpropagation method in the learning process. The number of layers and the hidden layer dimensions in this NN must be adjusted appropriately with the data and input dimensions. After learning the target users’ rating, each initialized user vector and item vector are improved according to each user profile and item characteristics. NCF [20] and temporal CNN for reviews based on the RS (TCR) [22] are examples of the NCF without neighbors concept.

NCF proposed three models: the GMF model, the MLP model, and the fusion of GMF and MLP called the neural MF (NeuMF) model. This work uses the one-hot vector to compute the model’s input, which is user and item representations. The GMF model is the MF method, which predicts rating value using an element-wise product between the user and item vector. The outcome of the GMF model is the predicted rating (φ^GMF) of the target user on the target item. The MLP model also uses user and item vectors. The user and item vectors are combined using concatenation to learn the interaction between users and items. φ^MLP is the interaction vector that is the result from MLP network. The NeuMF model is the model that combines the GMF model and the MLP model. The GMF and MLP compute the rating value and the interaction vectors, respectively. The outcome of the GMF and the interaction vectors from MLP are merged into one vector using a concatenation operation in each user and item pair. Then, these concatenated vectors are learned in the last NN layer to create the predicted rating value r_ui and labeled by the target user’s actual rating on the target item: (3) where h and σ denote weight and activation function of the NeuMF network output layer. r_ui is the predicted rating of the target user u on the target item i. Therefore, their work propose a framework that learns the rating and user’s interaction of NCF using only MF and the user’s interaction. However, there is no concept of neighbors in CF, which uses the neighbor’s preference for recommending items to the target user.

ONCF [21] improves NCF by learning the pairwise correlations between users and items by CNN instead of MLP. These correlations are the ratings of the users, given by the outer product between the users’ embeddings and the items’ embeddings. Their work aims to increase the efficiency of learning correlations between users and items. However, learning the rating matrix via CNN still lacks in leveraging the CF’s concept.

Because the user preference changes over time, the TCR model applies a time model in a CNN model. The TCR model consists of two networks: the user network and the item network, which execute the same process independently. Their work uses reviews from the users to extract the latent user profiles and item characteristics using the embedding method [27, 28]. Afterward, the user profiles and item characteristics are used as the input of the next layer, which is a convolutional layer and max-pooling layer. In the prediction step, both the user profile vector and item characteristic vector are used to compute the predicted rating vector using the MF technique (1). However, the comments or reviews can represent a user preference but cannot directly represent a user characteristic. Some of the comments that cannot represent a user characteristic include “fast delivery,” “Good quality bag,” and “Shoes are normally small on me so I went a size up and they fit great.” Moreover, this general framework does not consider the target user’s neighbors, which is a key CF technique.

2.1.2 Neural CF with neighbors’ concept.

Some researchers have applied the concept of neighbors into an NN such as NNCF [23], BPAM [24], the CMN [25] and SAMN [26].

The NNCF aims to integrate the neighbors’ information into the NCF model by concatenating the rating with the user’s neighbor information vector and the item’s neighbor information vector. These neighbor information vectors are performed by max-pooling each feature of all neighbor’s vectors. However, just adding the information using the concatenation operation into the ratings without using the similarity between the neighbor and the target user is not enough for the concept of CF technique.

BPAM proposed using the local weight and global attention weight to learn the rating of the target user. The local weight is the impact of the target user on each neighbor. The global attention weight is used as the impact on the target user on a set of neighbors. Both the local weight and global attention weight are combined and trained to predict the rating score. Although their work uses the similarity between the neighbors and the target user, it does not use the neighbors’ ratings into the prediction process as CF’s concept.

CMN tries to capture the user’s similarity and enhance the neighbor component in the NN attention mechanism. There are three steps: embedding, neighborhood attention, and rating prediction. First, the user history records are represented as a vector and form a user preference. Second, the new target user representation is obtained, which combines the neighbor component and the user embedding. The attention weights that infer the neighbor’s importance are fused into a NN layer to obtain the final neighbor’s representation. Finally, the target user’s rating score is produced via the MF concept using the target user’s and the target item’s representation. Then, the final neighbor’s representation and the target user’s rating scores are learned using a skip-connections NN. The CF’s prediction equation uses weighted average using the similarity among users and the neighbor’s rating scores where the similarity is used as a weight. However, CMN uses the neighbors’ representations to learn with the target user rating scores instead of the similarity level as in the CF concept.

SAMN proposes neighbor prioritizing, which considers the level of user preference. Because each neighbor of the target user has different expertise on different target items, Chen et al. [26] tried to create a user profile more accurately by fusing neighbors’ influence into the target user embedding. First, the user–item ratings are represented as user–item embeddings. The target user embedding and their neighbors embedding are combined through a joint embedding process to obtain an attention vector between the neighbor and the target user. Afterward, a fully connected layer is used to learn the attention score and normalize using a softmax function. To compute the friends’ vectors, the neighbor’s vectors are multiplied with their attention vectors dependent on the target user. Then, the friend-level attentions are computed and used to modify the target user embedding. Finally, the prediction score is computed using a MF between the modified target user embedding and the target item embedding.

Both current NCFs with friends, CMN and SAMN, give weight to the target user’s neighbor by combining them with the neighbor embedding. However, neighbor embedding is the representation of a user profile in terms of a numeric vector. Combining the weight with the neighbor embedding makes the user representation change, meaning it will cause the users’ representation to deviate from the real user profile and influence incorrect predictions. The neighbor’s weight should apply to the user’s rating for computing the target user’s predicted rating on the target item as in the CF rating prediction Eq (4): (4) where r(i, j) denotes the predicted rating of item j for the target user i, sim(i, n) is similarity between the target user i and user n, N is the set of neighbors, and r_n,j is the rating of user n given to item j.

The Table 1 shows the comparison of the neural based CF’s related works. “✓” represent the presence of the neighbor concept in each column and “X” represent the absence of the neighbor concept. “-” means that there is no comparison. In summary, PLF is an MF-CF model rather than Neural CF because their work applied only traditional CF with MF, which did not involve any NN in the model. AK-UCF utilized a NN to construct user’s and item’s embedding. However, a neighbor part is created using a clustering approach separately. Therefore, AK-UCF is a semi-Neural CF model. The NCF, ONCF, and NNCF applied the neural network with the CF technique but did not use the neighbors of the target user to predict the rating. BPAM, CMN, and SAMN proposed using the neighbors and performing the similarity between the neighbors and the target user differently. However, they did not create and apply the neighbors’ ratings in the prediction process as the CF’s prediction equation. Moreover, no research converts the neighbor’s preference range into the target user preference aspect. The proposed method converts the neighbor’s preference range into the target user aspect and apply the neighbor’s concept using the similarity value and the neighbor’s rating.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of the related works with respect to the neighbor concept.

https://doi.org/10.1371/journal.pone.0266512.t001

2.1.3 Rating conversion.

In the CF concept, the neighbors’ rating is used to predict the target user’s rating directly. However, the neighbors’ rating and the target user rating are in different ranges. Considering the neighbors and the target user preferences must use the same range. Thus, the method that converts the neighbor’s rating pattern into the target user’s rating pattern is called rating conversion. For example, suppose that the rating range is 1–5. User A gives ratings in the 1–3 range for “dislike” to “like,” while user B gives ratings in the 3–5 range instead. This means that “like” of A (rating 3 of A) equals “dislike” of B (rating 3 of B). Thus, using one user’s rating to predict another user’s rating directly is not good.

Although some researchers tried to solve the rating conversion issue using the normalization technique that maps every user’s rating into a range from 0 to 1, it is still not effective enough. If two target users have the same group of neighbors, both target users will receive the same predicted rating or the same recommendation. For example, target users A and B usually have normalization rates of 0.4 and 0.7, respectively, and the predicted ratings of both the target users on the target item are 0.8. Therefore, this target item should be recommended to user A rather than user B because the predicted rating of the target item is in the range that user A likes, while it is in the average range of user B. The W’s transpose function [29] is proposed to solve the rating conversion issue; it transposes the user’s rating into the target user’s aspect using a relation between both users’ rating patterns. There are four key terms in the W’s transpose function, which used to convert the user’s rating: original value, adjustment, confidence, and distribution. The original value is the rating that will be transposed. The adjustment part is the average difference of co-rated items between the user and the target user. The confidence term and distribution term are added to obtain more accurate results.

2.1.4 Region embedding.

Recently, a new embedding technique called the region embedding method that uses the words in the same region to obtain the word representation was proposed [30]. It is the assumption that one embedded word should not be in the same representation in a different region of the document. It is a representation of the continuous subsequence of the words in the document called the text region. Moreover, this method can solve the sparsity problem of the n-grams method. The text region, for example, given a sentence “The story is sweet and simple and easy to read,” a region of length five means “story is sweet and simple.” The region embedding method consists of three steps, as shown in Fig 1, to obtain the region embedding vector.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Context-word region embedding method.

https://doi.org/10.1371/journal.pone.0266512.g001

First, embed input words in the selected region to produce the target word embedding vector and local context unit (LCU), a word’s weight matrix in this region. Both the target word embedding vector and LCU are produced using the embedding method. Second, project the LCU into the target word embedding vector using the element-wise multiplication operation: (5) where is the projected word embedding of w_i+t in i_th word, is LCU, and is the embedding of word w_i+t. Afterward, the projected word embeddings, which are from the previous step, are combined using max pooling to perform the word representation.

2.2.1 Overview of the proposed method.

In region embedding, the word that is in the same region as the target word is called the LCU. For example, if the target word “Apple” is in the two regions with the LCU1 and LCU2 for region1 and region 2, respectively, the word “Apple” is represented as “Apple 1” and “Apple 2,” respectively. Because of the difference between the relation of the target word and each LCU, the representation of the target word is different in each region.

When applying the region embedding assumption with the CF concept, the target user is viewed as a target word. The neighbors of the target user are viewed as the LCU of the target word. Therefore, the target user will give the different predicted ratings for the two different groups of neighbors, similar to the “Apple 1” and “Apple 2” in the region embedding example (Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Context unit example.

https://doi.org/10.1371/journal.pone.0266512.g002

In CF, the concept of neighbors is integrated into the model in the prediction step. The neighbors’ rating scores and the similarity levels between the neighbors and the target user are used to predict the target user’s rating score as in (4). In the past, no NCF research used neighbors’ ratings in the prediction process as in CF. When applying a NN with the CF, two issues need to be considered: the similarities between the neighbors and the target user and the rating conversion. The region embedding assumption is used to solve the rating conversion issue, converting the neighbors’ ratings range into the target user rating range. The similarity issue is solved by capturing neighbors’ attentions to compute the similarity level between the neighbors and the target user.

In the case of similarity issue: the neighbors’ concept is the CF technique’s key idea, which uses the neighbors’ preferences to compute the similarity levels between the neighbors and the target user and the rating prediction. To compute the similarity, many techniques are used, such as cosine similarity and Pearson’s correlation, to determine the neighbors’ preferences. The similarity levels between the neighbors and the target user are used to weigh the neighbors’ ratings to predict the target users’ ratings. For example, a rating range of 1–5 for “dislike” to “like” and a similarity range of 0–1 for “not similar” to “similar.” The target user has two neighbors, N₁ and N₂, in Fig 3. The similarities between the target user and neighbors N₁ and N₂ are 1 and 0.5, respectively. If both neighbors gave a score of 5 to the target item, but the similarity value is different, the rating from N₁ and N₂ is transferred to the target user depending on the similarity level of N₁ and N₂. Therefore, the target user’s rating predictions via N₁ and N₂ are 5 and 2.5, respectively.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Example of similarity among users.

https://doi.org/10.1371/journal.pone.0266512.g003

In the rating conversion issue, because of different users having different rating ranges, the neighbors’ ratings are mapped from the neighbors’ rating range to the target user’s rating range. For example, the target user gives ratings in the range of 1–5 for “dislike” to “like,” while N₁ gives ratings in the range from 1 to 3 and N₂ gives ratings in the range from 3 to 5 instead. Suppose both neighbors gave scores of 3 to the target item, as shown in Fig 4. If the rating conversion is not considered, scores of 3 from the neighbors will map directly to the target user, which means neutral for the target user. However, N₁ wants to tell the target user that the target user should like the target item, which is a score of 5 for the target user, and the N₂ wants to tell that the target user should not like this item, which is a score of 1 for the target user. Therefore, the rating conversion should be considered in the model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Example of rating conversion issue.

https://doi.org/10.1371/journal.pone.0266512.g004

To solve these two issues, we propose three modules: the rating conversion module, the similarity module, and the prediction module. The rating conversion module projects the neighbor’s characteristics on each target user’s perspective view. In other words, the neighbors will be changed into the term of target user aspect, which is similar to the LCU concept of the region embedding. The similarity module captures the neighbors’ attentions to compute the similarity levels between the neighbors and the target user. The prediction module uses the rating conversion and the similarity module outcome to imitate the CF’s prediction equation. The architecture of the proposed model is shown in Fig 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. The proposed model architecture.

https://doi.org/10.1371/journal.pone.0266512.g005

2.2.2 Steps of the proposed method.

The proposed method consist of five steps: selecting neighbors, user–item embedding, converting neighbors into the target user aspect, crating similarity, and rating prediction.

2.2.2.1 Selecting neighbors. In the CF technique, using the neighbors’ concept to predict the target user’s rating on the target item is crucial. To select N neighbors of each target user, the group of rater who rated the target item are chosen. If the number of rater is less than or equal to N, all rater users are selected as neighbors. If the number of rater users is greater than N, the N rater are selected randomly to be the neighbors of the target user. Therefore, the outcome of this step is a list of N neighbors.

2.2.2.2 User–item embedding. After preparing the neighbors’ list, the inputs of the proposed method are the target user, the target item, and the list of neighbors. In each history record, the user and item pair are initialized in the same embedding size to represent the target user and the target item. The neighbors’ list of the target user is initialized as a matrix called the neighbors’ matrix, and each row is the neighbor’s embedding. If the number of the rater is less than N, the weights of all rater users’ in the NN model are initialized. The remaining neighbors’ weights are assigned zero. For example, the number of neighbors is set to 10. Therefore, 10 neighbors’ embeddings are initialized. Suppose the number of rater users is six, which is less than 10, then model’s weights of six neighbors are initialized. The remaining four weights are assigned as zero, meaning these four neighbors’ embedding will not be learned. The target user vectors, the target item vectors, and neighbors’ matrix will be adjusted to represent the target users, items, and neighbor characteristics using the target user ratings as a label of the model.

2.2.2.3 Converting neighbors into the target user aspect. To solve the rating conversion issues, the LCU concept in the region embedding [30] is applied. In region embedding, the LCU is the influence of each word in the region on the target word, which is used to obtain the target word representation in that region. In the same way, we apply the LCU with the neighbor concept by learning LCU via the target user ratings and use as the target user aspects on the neighbors to generate the neighbor representation in the target user’s perspective view. Therefore, we propose the rating conversion module in Fig 5, in which the neighbor’s preferences are converted into the target user aspect. First, the LCU of target user i (LCU_i) is initialized in embedding the size-by-number of the neighbors’ matrix. In Fig 5, suppose there are three neighbors and four elements of embedding size. Therefore, the dimension of the target user’s LCU is a four-by-three matrix. Afterward, the target user’s LCU on the neighbor n (LCU_{(i, n)}) is combined with the neighbors’ embeddings (u_n) using the element-wise multiplication to obtain the projected neighbors’ representations in the target user perspective (). (6) To compute the neighbors’ predicted rating scores in the target user aspect (r_(n,j)), these projected neighbors’ representations and the target item’s embedding (i_j) are integrated together using MF technique (7). (7) To adjust the target user’s LCU, neighbors’ embeddings, and other parameters in the model, the target user ratings are labeled and learned using the backpropagation algorithm.

2.2.2.4 Creating similarity. Usually, the similarity levels between the neighbors and the target user are computed from cosine similarity or Pearson’s correlation equation on a pair of the target user and each neighbor. In this work, the similarity levels between the neighbor and target user are attentions between two users. To compute attention among users, a pair of the target user vector (u_i) and the neighbor vector () are combined using a dot product. (8)

Afterward, the softmax function (9) is used to adjust the similarity value into the range [0, 1]. This step’s architecture is shown in the similarity module in Fig 5. (9)

2.2.2.5 Rating prediction. In the CF approach, the concept of neighbors is integrated into the system in the prediction step, in which the neighbor’s ratings are used to predict the target user’s rating. The same as the CF approach, the rating prediction Eq (10) is imitated into the proposed method in this step using the neighbor’s predicted rating scores and the similarities between the target user and their neighbors. As the rating prediction formula, the neighbors’ rating (r(n, j)) needs to be weighed because each neighbor has a different similarity level toward the target user. Therefore, the target user’s predicted rating scores are computed using a weighted average on the neighbors’ ratings where the similarity levels are used as a weight. The result of the proposed method is the predicted rating of the target user i on the target item j (r(i, j)). The prediction module in Fig 5 is the architecture of this step. (10)

Finally, we calculate the time complexity for one epoch to train the NN. Due to the proposed method performed by three layers, the time complexity of feedforward and back-propagation is O(e_n e_cn t), where e_n, e_cn, and t are the neighbor embedding size, the neighbor in the target user aspect embedding size and the number of neighbors, respectively. In this work, the neighbor embedding and the neighbor in the target user aspect embedding are the same sizes. Due to training the model, the number of epochs in each dataset is different. The time complexity of our proposed method also depends on the number of epochs. Therefore, the time complexity of the proposed method depends on the neighbor embedding size, the number of neighbors, and the number of epochs.

3.1.1 Data preparation.

To evaluate the performance of the proposed method, three public datasets are used:

• MovieLens 1M: [31] This dataset is collected by grouplens. It contains one million movie ratings, is a stable benchmark dataset, and is widely used in research experiments. It contains the user ID, movie ID, users’ ratings, and timestamp. Moreover, grouplens provides at least 20 records per user.
• Epinions: [32] This dataset is a who-trust-whom online social network that contains product ratings and reviews. It contains the product name, category, and timestamp.
• Yelp: [33] This dataset contains users’ personal data and reviews, ratings, pictures, and details of business and services in four countries. It is released for the academic challenge and teaches students about databases.

The rating range of MovieLens 1M, Epinions, and Yelp dataset is [0.5,5]. Each dataset consists of four basic attributes: userID, itemID, rating, and timestamp. The number of records, users, and items of each dataset are shown in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The number of records, users, and item in each dataset.

https://doi.org/10.1371/journal.pone.0266512.t002

To ensure that each user has enough records represented as a vector, the users who have more than twenty records are selected in the preprocessing step. The number of records, users, and items after preprocessing are shown in Table 3. The MovieLens dataset provides at least twenty records per user. Thus, the number of records, users, and items in Table 3 are the same as in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. The number of records, users, and items in each dataset after preprocessing.

https://doi.org/10.1371/journal.pone.0266512.t003

The statistics of each dataset have shown in Table 4, which consist of Mean, Median, Mode, and Standard Deviation (SD). From Table 4, the highest mean value goes to the Epinions dataset. That means the Epinions dataset contains the highest number of high rating scores compared with the Movielens and yelp datasets. The rating scores of all datasets have the same median value, which is four. Most of the rating scores in the Epinions and Yelp datasets are five, while the Movielens dataset is four. When comparing a standard deviation, the Yelp dataset contains the highest rating dispersion compared to the Movielens and Epinions datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Statistic of each dataset.

https://doi.org/10.1371/journal.pone.0266512.t004

3.1.2 Experimental settings.

To evaluate the dataset, each dataset is randomly selected in the ratio 80:10:10 for training:validation:testing sets. The baseline parameters are used as in the corresponding papers. The proposed method’s batch size was tested in [32, 64, 128], and the learning rate was tested in [0.01, 0.05, 0.1]. To demonstrate the impact of the number of neighbors and embedding size, we fixed and evaluated the proposed model at nDCG@k for k ∈ {5, 10, 20, 30, 40, 50}. In this work, we tried to use an embedding size of 8, 16, 32, 64 and neighbor size of 10, 20, 30, 40, 50.

This work aims to apply a NN into the CF concept, which considers similarity levels between the neighbors and the target user and rating conversion. To optimize the model, we use the adaptive momentum (Adam) [34] estimation optimizer, which is widely used in current works and requires small memory. The Adam optimizer is a combination of root mean square propagation gradient descent (RMSprop) [35] and stochastic gradient descent (SGD) [36]. In the learning process, the Adam optimizer computed adaptive learning rates for each parameter and the exponentially decaying average of past gradients. We used the loss function in a multi-class classification task called categorical cross-entropy to minimize the prediction error and learn the model parameters: (11) where C denotes the number of output classes, which is a number of rating classes. In this work, C is set at 10 or 5 depending on the range of rating in each dataset. y_(i,j) and r_(i,j) are the actual rating and the predicted rating of user i on target item j. After evaluating the impact of the number of neighbors and embedding size of the proposed method, we compare the proposed method with the other two methods using nDCG@k, HR@k for k ∈ {5, 10, 20, 30, 40, 50}, Precision, Recall, and RMSE.

3.1.3 Evaluation metrics.

To evaluate the proposed method, there are two types of evaluation metrics: ranking-based evaluation and prediction accuracy. Two ranking quality metrics are used in this work: nDCG and HR. In addition, we use precision, recall, and RMSE to indicate the prediction accuracy.

3.1.3.1 Ranking-based evaluation. The nDCG is one of the most popular metrics that evaluate the ranking quality using relevant score. The nDCG equation defined as follows: (12) where p is a particular rank position, rel_i is the relevance of recommendation at position i, and |REL_p| is the list of items ordered by relevance in the corpus up to position p.

The HR indicates the accuracy of recommendation top k. If the item in the top k recommendation list matches the item that the target user has rated in top k, it means “hit.” The number of hits divided by the number of all users in the test set is the HR value. (13)

3.1.3.2 Prediction accuracy. To evaluate the prediction accuracy, we use three metrics: precision, recall, and RMSE. Precision is a measure of exactness. It is the proportion of the recommended items in the top k set that are relevant, which is the number of the recommended items that are relevant divided by the total number of recommended items. The precision is defined as follows. (14) The recall is the proportion of the relevant item on a total number of relevant items, which is the number of the recommended items that are relevant divided by the total number of relevant items. (15) The predicted ratings that are higher than three scores are defined as relevant in this work.

Another way to evaluate the rating prediction accuracy is by measuring the error of the predicted rating. Therefore, RMSE is used. To calculate the RMSE, the predicted rating (r(i, j)) is compared with the actual rating (). The summation of the error squared values is averaged over all predictions to obtain the overall RMSE: (16) where N is the total number of the prediction.

3.1.4 Experimental results.

The number of neighbors, who have rated the target item, is important in this work. Traditional CFs use the similarity between the neighbor and the target user to weigh the neighbor’s ratings. Therefore, the neighbors of the target user are essential in the CF research. In the proposed method, the neighbors are selected by two criteria: the users who have rated the target item (raters) and the similarity value of more than zero. In section 2.2.2.4, the attentions between the neighbor and the target user are normalized using the softmax function and the threshold to create the similarity value between the neighbor and the target user. The similarity between the neighbor and the target user in traditional CF’s technique is in the range [0, 1]. In this work, we select the raters who have similarity more than zero as the neighbors. The similarity between the neighbor and the target user is in range [0, 1] as in CF technique. This similarity value indicates how similar the neighbor is to the target user. Suppose the similarity value of the neighbor closes to one means this neighbor is most similar to the target user. In contrast, if the similarity value is close to zero, this neighbor is a low-quality neighbor. However, there is a large number of raters in each dataset. Using all raters as the neighbors can make the incorrect prediction because some neighbors are the noise of recommendation. In order to find the effectiveness of the number of neighbors, this work evaluates two types of neighbors usage: N neighbors and all neighbors. The N neighbors are the randomly selected N raters as neighbors. In contrast, all neighbors use all users who have rated the target item to be the neighbors.

3.1.4.1 N neighbors. The proposed method solves the two issues: the similarities between the neighbors and the target user and the rating conversion. The neighbors’ concept is considered and applied to the NN model, in which the neighbors are used to predict the preference of the target user. Because the similarity issues are directly implicated with the number of neighbors, it is an essential hyperparameter that needs to be tuned. The embedding size of the target user, neighbor, and item representations are other essential hyperparameters used to represent the user–item interaction profiles via ratings. The MovieLens 1M dataset’s experimental results are shown in Fig 6. Fig 6 shows the changes in the experimental results when using different embedding sizes and a different number of neighbors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Experimental results of the Movielens 1M dataset for the proposed method varying the embedding size and neighbors.

[a] nDCG@5. [b] nDCG@10. [c] nDCG@20. [d] nDCG@30. [e] nDCG@40. [f] nDCG@50.

https://doi.org/10.1371/journal.pone.0266512.g006

Fig 6[a]–6[f] shows that the embedding size of 8 obtained the highest nDCG results when the numbers of neighbors were 10 and 30 when comparing with the embedding sizes of 16, 32, and 64. Even though the embedding size of 16 provided the highest performance in Fig 6[a] when there were 20 neighbors, 10, 30, 40, and 50 neighbors gave lower nDCG results than the other three embedding sizes. In the same case as the embedding size of 16, the embedding size of 64 obtained the highest nDCG results when the number of neighbors was 50 in Fig 6[a]–6[e]. However, 10, 20, 30, and 40 neighbors at the embedding size of 64 obtained lower nDCG results than the other three embedding sizes. Although the embedding size of 32 obtains the same tends as the embedding size of 8, the embedding size of 32 provides a lower nDCG result than the embedding size of 8. The overall nDCG results obtained from embedding size 8 are greater than the other three embedding sizes, especially at a neighbor size of 30. In terms of the neighbor parameter, most of the nDCG results increase when the number of neighbors reaches 30. Afterward, the results decrease. The result shows that increasing the number of neighbors can improve the rating prediction accuracy. According to the experiment results, only the first 30 neighbors are high-quality neighbors, and the number of neighbors more than 30 neighbors are low-quality neighbors. If the number of neighbors is too high, the low-quality neighbors decrease the nDCG. Therefore, we selected an embedding size of 8 and neighbors 30 to compare with the CMN and SAMN on the MovieLens 1M dataset.

We tried to experiment with an embedding size of 8, 16, 32, and 64 and the number of neighbors 10, 20, 30, 40, and 50 on the Epinions and Yelp datasets. Moreover, the number of the neighbor parameter is varied in the same way as in the MovieLens 1M dataset on both datasets. The overall nDCG ranking results from the Epinions and Yelp datasets that obtain the best results are at embedding sizes of 64 and 32, respectively. Thus, the Epinions dataset at an embedding size of 64 and the Yelp dataset at the embedding size of 32 are used to compare with the proposed method.

The comparison results of the proposed method’s results, latent factor models, and the current NCFs with friends using 30 neighbors at embedding sizes of 8, 64, and 32 on the MovieLens 1M, Epinions, and Yelp datasets using nDCG are shown in Table 5 and Fig 7[a]–7[c]. The HR experimental results are shown in Table 6 and Fig 7[d]–7[f]. The proposed method achieves significantly higher results than SVD, NMF, CMN and SAMN at all k. Although some of the proposed method results are slightly lower than the SAMN method, the proposed method takes much less processing time than the SAMN method, approximately three hours less on the MovieLens 1M dataset. The Epinions dataset is a small dataset when compared with the other two datasets. Therefore, this dataset does not have enough records to experiment on nDCG@40 and nDCG@50 after preprocessing.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Experimental results graph comparison on the ML-1M, Epinions, and Yelp datasets using the nDCG and HR.

[a] nDCG ML-1M dataset. [b] nDCG Epinions dataset. [c] nDCG Yelp dataset. [d] HR ML-1M dataset. [e] HR Epinions dataset. [f] HR Yelp dataset.

https://doi.org/10.1371/journal.pone.0266512.g007

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison of the experimental results on the ML-1M, Epinions, and Yelp datasets using nDCG.

https://doi.org/10.1371/journal.pone.0266512.t005

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 6. Comparison of the experimental results on ML-1M, Epinions, and Yelp datasets using the HR.

https://doi.org/10.1371/journal.pone.0266512.t006

The experimental results between the proposed method and the other methods using prediction accuracy metrics on three datasets are shown in Table 7. The proposed method achieves significantly higher performance than SVD, NMF, CMN, SAMN, and the proposed method without the rating conversion module when compared using precision and recall on all datasets. Moreover, the proposed method also obtains the lowest errors on the RMSE metric. Therefore, converting the neighbor’s preference range into the target user aspect and indicating the user similarity through the neighbors’ attentions can provide more accurate experimental results.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 7. Predicted accuracy experimental results.

https://doi.org/10.1371/journal.pone.0266512.t007

To evaluate the effectiveness of the rating conversion module, we compare the proposed method with the proposed method without rating conversion. The experimental results are shown in Tables 5–7 and Fig 7. The proposed method achieves higher efficiency than the proposed method without the rating conversion at all evaluation metrics because the neighbors’ preference ranges are converted into the target user aspect. Moreover, the proposed method without the rating conversion can provide higher performance than the CMN but lower than the SAMN. Therefore, converting the neighbors’ ratings into the target user’s perspective view can make the prediction more accurate than not converting the neighbors’ ratings.

In additional, we compare the proposed method with latent factor models. The experimental results have shown in Tables 5–7. From the results show that the proposed method is perform better than the SVD and NMF, which are the latent factor model in both ranking-based evaluation and prediction accuracy.

3.1.4.2 All neighbors. From selecting N neighbors as neighbors, the 30 neighbors obtain the highest performance. Therefore, the result of the 30 neighbors is used to compare with all neighbors in this subsection. Table 8 show the evaluation results, which compare the nDCG results of the 30 neighbors (N = 30) and all neighbors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 8. The experimental results of using N neighbor and all neighbor.

https://doi.org/10.1371/journal.pone.0266512.t008

The results from ML-1M and Yelp dataset with using 30 neighbors obtain higher than using all neighbors in all evaluation metrics. While the Epinions dataset with all neighbors obtain the higher than using 30 neighbors. From Fig 8[a] and 8[c], it can be seen that almost similarity values of ML-1M and Yelp dataset are close to zero, which means almost neighbors are less similar to the target users in both dataset. In contrast, almost the similarity values in the Epinions dataset are close to one in Fig 8[b]. Therefore, the neighbors from the Epinions dataset have a high quality to use in recommendation than the ML-1M and Yelp datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Similarity values between the neighbors and the target user of three datasets.

[a] ML-1M dataset. [b] Epinions dataset. [c] Yelp dataset.

https://doi.org/10.1371/journal.pone.0266512.g008

3.2.1 Data generation setting.

In this work, the rating range of both full and partial rating matrices are set to range 1-5. In order to generate the datasets, there are four main parameters, which are the numbers of user, item, raters, and rating score. We desire to evaluate three types of distributions of the full rating matrix:

• Normal rating distribution: the dataset where the number of rating in each rating score is equal. Fig 9[a] is the example of the normal rating distribution, which each rating score has one rating.
• Skewed right rating distribution: the dataset where the number of high rating scores is more than the number of low rating scores. Fig 9[b] shows the example of skewed right rating distribution, which the number of score 3, 4 and 5 are more than score 1 and 2.
• Skewed left rating distribution: the dataset where the low rating score is more than the high rating score. The example of skewed right rating distribution shows in Fig 9[c]. The number of rating score 1 and 2 are more than score 3, 4 and 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. The example of synthetic datasets.

[a] Normal rating distribution [b] Skewed right rating distribution. [c] Skewed left rating distribution. [d] Partial rating distribution.

https://doi.org/10.1371/journal.pone.0266512.g009

Moreover, the size of each rating distribution depends on k of nDCG matrics to compare the effect of dataset size. In this work, we evaluate nDCG @k for k ∈ {5, 10, 20, 30, 40, 50}. Therefore, there are six datasets size in each distribution. The data in dataset is divided into ratio 80:20 for train:test sets. Suppose k = 5, the minimum number for user–item interaction that can evaluate using k = 5 is twenty five interaction. Therefore, we use the number of user, item, rater and total number of rating score equal to twenty five.

We simulate the normal distribution partial rating dataset, which more similar to the real-world dataset. The partial dataset is the dataset that the users have rated some items and has more additional rating data than the full rating dataset. The example of partial rating dataset has show in Fig 9[d]. The partial rating matrix is generated by using 1000-by-1000 rating matrix and fifty raters per items.

Table 9 shows the parameter setting of all synthetic dataset. The dataset gen1 to gen6 are the normal data distribution with different size. The datasets gen7 to gen12 are the skewed right data distribution and datasets gen13 to gen18 are the skewed left data distribution. The dataset gen19 is the partial dataset that has two hundred and fifty raters similar to the gen6. However, gen19 has more rating patterns than the gen6 dataset because the target user does not need to rate all items in the dataset.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 9. The parameters of simulating data.

https://doi.org/10.1371/journal.pone.0266512.t009

3.2.2 Experimental results.

In this work, nineteen datasets are evaluated to study the three different data distributions and different sizes of the datasets. Table 10 shows the experimental results of all datasets by using nDCG. From Table 10, the proposed method obtained the highest nDCG results when compared with the baselines in terms of the ranking list.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 10. The nDCG experimental results of synthetic data.

https://doi.org/10.1371/journal.pone.0266512.t010

In terms of data distribution, the experimental results show that the skewed right rating distribution dataset obtained the highest nDCG results compared to the normal rating distribution and the skewed left rating distribution. Since most rating scores in the skewed right data distribution are high rating scores, these datasets have many positive preferences. In contrast, the skewed left data distribution has a lot of negative preferences. Usually, making recommendations is more about suggesting things users like than suggesting things users do not. Therefore, using positive preference is more effective in recommendation than using negative preference. The normal rating distribution datasets have an equal number for each rating score, which means a number of positive preferences equal negative preferences. Thus, the normal rating distribution datasets obtained the lower nDCG result than the skewed right rating distribution dataset and higher nDCG results than the skewed left rating distribution dataset.

Regarding size comparison, the small dataset obtains higher nDCG than the large dataset because simulated datasets depend on k. It means the small dataset is less k, the large set is large k. Therefore, using a small number of k has a greater chance of ranking correctly than the large k.

The full normal distribution rating matrix, gen6, is the maximum number of raters used to evaluate in this work. Therefore, we created gen19 in a normal distribution with the number of raters equal to gen6. The dataset gen19 obtained a higher ranking quality than gen6. Because the partial rating dataset has more different patterns of interactions than gen6, which is gen19 has additional data in learning. Therefore, gen19 can predict the recommended item better than gen6.

The HR metrics use the list of the rating prediction in test set to evaluate how many ratings in k-sized list of ranked items are matched with the actual rating ranked items. Due to the number of interactions in simulated datasets are generated by k, the HR ranked items results of all simulated datasets is equal to k. It means all of the predicted ratings are used in HR evaluation. Therefore, the HR results of all simulated datasets is one.

The accuracy of experimental results using Precision, Recall, and RMSE are shown in Table 11. The experimental results of the proposed method obtain the highest prediction results when compare with the two latent factor models and two current NCF with friends.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 11. The accuracy experimental results of synthetic data.

https://doi.org/10.1371/journal.pone.0266512.t011