Shilling attack detection for collaborative recommender systems: a gradient boosting method

1.   Introduction

In an age of information overload, the amount of data that far exceeds our processing power has permeated our society. Recommendation systems are one of the most important tools for solving information overload due to their unique technical advantages. In recent years, the application of new machine learning and deep learning techniques ^[1,2,3] to the recommendation systems has continued to receive attention from many academic researchers. Collaborative filtering ^[4] recommendation systems (CFRS) help people to make personalized choices while alleviating the problem of information overload. The subject of this paper is a recommendation system based on collaborative filtering techniques. One of the foundations for the implementation of collaborative filtering technology is the user profile file in the rating matrix. There is a positive correlation between the higher order of magnitude of the user profile and the output of the recommendation system ^[5]. Therefore, this requires the recommendation system to have strong open nature to facilitate different user registration ratings.

Malicious attackers inject a percentage of fake users into the recommendation system through certain methods, with the ultimate goal of changing the rating of targeted items and gaining illicit financial gain ^[6]. The above operations can lead to system misclassification, causing the prediction score of the items to change and producing the recommended results expected by the malicious attacker. Past literature has also referred to profile injection attack as shilling attack, this paper takes the latter terminology. Shilling attacks can interfere with the normal output of the recommendation system, or even cause it to crash as a whole. It is important for recommendation systems to perform the task of shilling attack detection in a reasonable and efficient way.

The attributes of all user profiles in a collaborative filtering recommender system can only be normal or fake users. Derive reasonable features and training high-performance classifiers are of great significance in improving the precision of shilling attack detection. Some past attack detection methods were used to analyze from the perspective of user ratings. But when a malicious attacker employs certain obfuscation techniques, the rating patterns of fake users and normal users will become similar. Some shilling attack detection algorithms that use features extracted from user rating values can cause misclassification for normal users. At the same time, shilling attack detection methods that rely on another single perspective also suffered from low precision problems.

To counter the disadvantages of past methods, we propose the XGB-SAD model based on a double-view and the mathematical principle of gradient boosting. First, we analyze it from the point of view of user rating time. The time window left for the shilling attacker to complete the attack profile injection is very short due to the time cost. However, the average interval between ratings of two items by normal users is generally not less than the length of an item. The above foundation provides the basis for solving the problem of shilling attack detection from a rating time perspective. Then, we analyze it from the point of view of the way users rating item. In order to eliminate the anomalous effects of user rating values, this paper analyses the problem in terms of the boolean values of the user rating items. Based on the above double-view, we build the TPUS collection. Thirdly, related literature shows that single classifiers are less effective in detecting shilling attacks. To address this limitation, our method uses eXtreme Gradient Boosting to perform heuristic iterative optimization of the model's objective function. Then, we use the idea of ensemble learning to integrate multiple sets of base classifiers into strong classifier. Finally, we perform multi-group experiments. The results demonstrate that XGB-SAD outperforms the comparison methods in terms of small-scale attack detection and overall detection.

In Section 2, we introduce the background of the paper; In Section 3, we introduce theoretical preparation; In Section 4, we present our approach. In Section 5, we performed an experimental evaluation; In the final section, we present a comprehensive summary of the article and describe future work.

2.   Background and related work

2.1.   Attack profile and shilling attack model

Malicious attackers inject fake user ratings into the rating matrix to change the target item's recommended frequency. We refer to the above actions of malicious attackers as shilling attack. The attackers will choose different types of shilling attacks depending on the specific operating environment and knowledge base. In the push attack, attacker's ultimate goal is to improve the attack item's recommended frequency. In the nuke attack, attacker's goal is the opposite. Table 1 lists the general forms used in rating matrix and shilling attacks. The user's rating values for items range from 1 to 5, with "0" representing no rating, "1" representing the lowest preference for items and "5" representing the highest preference for items. Taking push attack as an example, suppose the objective of the CFRS is to predict the rating value of ${user}_{2}$ to ${item}_{6}$. The principle underlying the technical implementation of the CFRS is to predict the rating of unknown items based on the similar nearest neighbours of the user. Before the push attack, the similar nearest neighbor of ${user}_{2}$ is ${user}_{1}$ and the system's prediction score is 1. The probability that the final recommendation list contains ${item}_{6}$ is very low. After the push attack is executed, the malicious attacker injects the attack profiles of ${attacker}_{1-4}$ into the system database. Then, the similar nearest neighbors of ${user}_{2}$ will become ${attacker}_{1-4}$ and the prediction score of the recommendation system will become 5. There is a high probability that the system will recommend the ${item}_{6}$ to the ${user}_{2}$. The attacker has successfully executed the push attack to reverse the system's interest in ${user}_{2}$'s items.

Malicious attackers choose different rating items depending on their purpose. The collection of these ratings and their parameters is called an attack profile. Table 2 shows the specific components used in attack profile. ${I}_{S}$ is a collection of items selected to be scored according to the specific needs of shilling attackers. In some attack models the ${I}_{S}$ is not a necessary component. ${I}_{F}$ refers to the filler items. The purpose of its existence is to more closely match the way real users rate their behaviour. ${I}_{\varnothing }$ refers to the collection of blank rating value items. ${I}_{T}$ refers to the target item. There can only be one target item in this paper.

Table 3 lists the parameters used in shilling attack models.

● Random attack: There is no ${I}_{S}$ in this attack model. The selection of ${I}_{F}$ and the rating values are random. ${I}_{T}$ is set to ${r}_{max}$ in the push attack.

● Average attack: There is no ${I}_{S}$ in this attack model. The value of ${I}_{F}$ is set to ${r}_{avg}$ in the system. The value of ${I}_{T}$ is set to ${r}_{max}$ in the push attack.

● Bandwagon attack: The value of ${I}_{s}$ is set to ${r}_{max}$ in this attack model. The ${I}_{F}$ items in bandwagon attack are selected from the Top-N items in the recommendation system. ${I}_{T}$ is set to ${r}_{max}$ in the push attack.

The above models are the most widely used and impactful shilling attack models. Correct analysis and detection of the above three attack models are important for our understanding of other attack models.

2.2.   User rating value

The rating matrix is the core data source for collaborative filtering recommendation systems. The user ratings behind the rating matrix represent the rating habits and rating preferences of normal users. The malicious attacker generates the user's rating by constructing a relevant attack model in bulk to simulate it. Items that are of real interest to malicious attackers are known as target item. In various shilling attack models, the target item is rated as an anomalous extreme score (highest or lowest score). However, other items are scored as average or random. Attack detection from the above perspective was the initial thinking of the researchers. The related researchers analyzed the differences between normal users and fake users in terms of user rating values and extracted classification features. The following equations show the mathematical definition of the rating matrix Eq (1) and some key classification features extracted from the user rating values in the rating matrix Eqs (2)–(7).

Rating matrix

Definition:

Rating deviation from mean agreement

Definition:

Weighted degree of agreement

Definition:

Weighted deviation from mean agreement

Definition:

Mean variance

Definition:

Filler mean variance

Definition:

Filler mean target difference

Definition:

2.3.   Shilling attack detection

Among the large group of technologies for recommender systems, CFRS is the most mature, widely implemented and theoretically well-developed technique. The technique of collaborative filtering is based on the following idea: In the rating matrix, the specific set with the most similarities to the target user is generated in a certain way. This user group is used as the basis for predicting the target user's rating value for the unrated items. Then the above results are combined to complete the recommendation to the target user. CFRS have important applications in the business world. It solves the problem of information overload for consumers and significantly reduces the economic costs for both sides of the transaction. However, the existence of shilling attacks seriously hinders the proper functioning of recommendation systems.

Shilling attackers influence the output of the CFRS by inserting fake users into the rating matrix within the database. The existence of shilling attack poses a serious risk to the security and stability of the CFRS. If not dealt with in a timely manner, the financial interests of both the recommendation service platform and the consumer can be greatly compromised. As a result, research on shilling attacks has received a lot of attention and has become a hot topic in the recommendation systems. A well-designed shilling attack detection method maintains the stability of the overall system and protects the property of both the recommendation system service provider and the consumer. Based on the different perspectives and concepts involved, a variety of detection algorithms have been designed by those involved. There are three main research directions for attack detection algorithms.

In the supervised learning, Chirita et al. ^[7] were among the first researchers to focus on shilling attacks. They first proposed the RDMA attribute and used it as the basis for a shilling attack detection algorithm. These pioneering studies provided a good basis for subsequent researchers. Burke et al. ^[8] extracted these detection features from the user profile and then combined them with the KNN algorithm model to perform the task of attack detection. Williams et al. ^[9] proposed several detection features and combine them with several machine learning algorithms. Specifically, the SVM has the best overall results. Tang et al. ^[10] proposed three detection factors based on rating interval as a basis for the task of shilling attack detection. Xia et al. ^[11] used the slope and first order derivative of the time series rating values to dynamically divide the time interval as a way to detect anomalous rating items. The above methods for identifying anomalous users or anomalous items based on rating time are mainly used to detect a large number of attacks with concentrated rating time, but the above methods are not effective for attacks with decentralized injection. Yang et al. ^[12] proposed three new detection features based on rating fill rate. Combining 15 existing detection features such as WDA and RDMA, the Re-scale AdaBoost algorithm is used to detect malicious users. Wu et al. ^[13] selected the features that work best for a specific attack type from many shilling attack features as detection features by means of feature selection for different shilling attack types. They also proposed two detection methods based on classical machine learning algorithms. Li et al. ^[14] proposed a method. The method uses statistical analysis of item popularity, which in turn reveals the difference in popularity distribution between fake and normal users. In the semi-supervised learning methods, Wu et al. ^[15] used an expectation max method to filter the rating features proposed by Williams. The method is effective in improving the detection accuracy due to the use of a more efficient subset of features. In the unsupervised learning approach, Mehta et al. ^[16] proposed unsupervised learning methods PCA-SelectUsers to detect malicious fake users. More importantly, this detection method requires certain information to be obtained in advance, such as the specific size of the attack, in order to intercept the attack situation based on a ranked list of principal components. Yang et al. ^[17] constructed an unprivileged user relationship graph, used graph mining methods to calculate user similarity, and used a clustering algorithm to detect some suspicious users. Finally, the attacking users were further screened based on target item analysis. This threshold value is difficult to determine in practical applications.

3.   Preliminaries

3.1.   User rating time

Malicious shilling attackers change the system recommendation frequency of target item with the aim of increasing their illegal financial interests. Because of the specific implementation principle of collaborative filtering technology, this malicious change can only be accomplished if the shilling attacks reach a certain size. The time cost of the shilling attacker makes the difference between the injected fake user and the normal user. Therefore, the number of ratings of the attack profile increases significantly in a relative unit of time. The shilling attack on recommendation systems is a short-term utilitarian act. This short-term behaviour is extremely evident in the rating intervals. The relative rating time interval for a normal user of an item is not less than the length of a movie. However, the relative rating interval ^[18] for a malicious shilling attacker will be significantly less than that of a normal user. Unlike normal users, the purpose of the shilling attacker is to gain illegal financial interests. Therefore, the economic cost is an important consideration for the attacker. The cost of time can have a very significant impact on the expected economic benefit of a malicious attacker, which is why there is a difference in the user rating time ^[19] of the fake user and the normal user. On the basis of the above, we have defined A Eq (8), B Eq (9) and C Eq (10). TPUS-1 in the TPUS set is defined in Eq (11).

A. Collection of user rating time (UTC)

Definition: The elements in the set are the timestamps of the user's rating items, sorted in descending order. That is, the higher the serial number of the element, the closer the element's rating time is to the current time. (u refers to a particular user and n refers to the n items rated by this user)

B. Maximum interval of user rating time (UTM)

Definition: UTM is the subtraction of the first and last element values in the user's UTC set. (n refers to elements in the user's UTC)

C. Aggregation index of user rating time (IUT)

Definition: IUT refers to the ratio of the user's UTM value to the total number of rated items ${N}_{u}$.(u refers to a particular user)

TPUS₁: Relative aggregation index of user rating time (RIUT)

Definition: RIUT is the first component of the TPUS collection and is used to describe the relative degree of aggregation of rating time. $\overline{UTM}$ is the average of the UTM values of the users within the database and $\overline{N}$ is the average of all user's rated items.

3.2.   User rating item

Past methods have mostly analysed the rating matrix from the single perspective of user rating values. However, there are a number of problems with the above methods. When recommendation system information is obtained through public or semi-public channels, a malicious attacker will use the above basis and various obfuscation techniques to design an attack model. The attack profile generated on the basis of this attack model will be very similar to the normal user's rating value habits. As a result, such shilling attack detection methods based on rating values can misjudge normal and fake users, affecting the accuracy of the detection algorithm. In order to eliminate the anomalous effects of user rating values, this paper analyses the problem from the user rating items.

In the rating matrix of a recommendation system, users select items based on preference and assign them different rating values. In this paper, we are not concerned with the specific rating values of the users, so we booleanize the users' rating values. The URB is defined in Eq (12). URB only cares if the user has a rating for the item, reducing the potential negative impact of obfuscation techniques on the rating value. The user rating matrix is transformed using Eq (12) to convert the original user rating values to Boolean values, and the transformed rating matrix is called the URB matrix. In the URB matrix, "1" indicates that the user has a rating action for the item and "0" indicates that the user does not have any rating action. Based on the user's URB value, we define the user's TCIB value in Eq (13). For an item in the URB matrix, TCIB calculates the sum of all non-zero URB values for the column in which the item is located. The TCIB value for each item represents how many times the item has been rated and how well it has been accepted. On the basis of the above, we have defined F Eq (14), TPUS-2 Eq (15) and TPUS-3 Eq (16).

D. Boolean values of user rating (URB)

Definition: To eliminate the effect of user rating values, definition URB booleans the user's rating values. ${R}_{ij}$ represents the rating value of ${user}_{i}$ to ${item}_{j}$ in the rating matrix ${R}_{m\times n}$. The specific definitions are as follows:

E. The coefficient of item boolean (TCIB)

Definition: TCIB is the sum of the URB values of all users in the column in which the item is located in a rating matrix (${R}_{N-F}$) that does not contain fake user. The TCIB value for j is defined as follows.

F. The set of user rating boolean item (UBS)

Definition: Among user's rating items, if an item has a TCIB value not equal to "0", it is placed in the UBS collection. The elements in the set are arranged in ascending order. The user's UBS is defined as follows.

TPUS₂: Mean Index of User boolean (MIUB)

Definition: In the user's UBS collection, the elements in the collection are first summed in order from the beginning to the end. The ratio of the summed value to the total number of users in the set is defined as ${TPUS}_{2}$.

TPUS₃: Sub-Range Index of User boolean (RIUB)

Definition: In the UBS collection of user u, subtract the value of the penultimate element from the value of the second element in the collection. n refers to the maximum number of elements in the UBS set.

3.3.   Ensemble learning and gradient boosting

Ensemble learning refers to a theoretical idea in which multiple base classifier models are combined with the ultimate goal of obtaining a more comprehensive model. Ensemble learning is based on the idea that even if a few weak base classifiers output incorrect predictions, most of the other base classifiers can correct the bias so that the model outputs the correct results. Compared to methods using only a single classifier, ensemble learning methods can continually minimize the error of the base classifier and output a more comprehensive and robust model. There are four common integration learning methods, namely boosting, bagging, stacking and blending. This paper focuses on the first type of method. Boosting is an important branch of ensemble learning. Specifically, each iteration of the boosting model is accompanied by the generation of a new classifier that focuses on the parts that were in error during the previous iteration. The new classifier generated by the boosting model is combined with the classifier from the previous iteration, and the process is repeated to achieve the desired learning goal and result in a more robust classifier model.

Gradient boosting is one of a large group of boosting algorithms that can be used in machine learning for regression, classification and sorting tasks. The basic principle is as follows: the negative gradient of the loss function in the current mathematical model is first calculated and used as the basis for training the next newly added weak classifier. The trained weak classifiers are then continuously accumulated into the existing model. Gradient boosting often chooses a decision tree (usually a CART tree) as the base learner, a method known as GBDT. There are many advantages of GBDT, such as robustness of the model, high accuracy, ability to handle non-linear data, etc.

eXtreme Gradient Boosting ^[20] is a specific implementation of GBDT and improves and enhances it in many ways. Since its invention, the XGBoost algorithm framework has boasted excellent performance (e.g. high accuracy rate, wide applicability, etc.) in various machine learning competitions and has gradually gained the attention of related researchers. Essentially, the idea of the XGBoost algorithm framework is ensemble learning, which is an advanced and improved version of the GBDT algorithm. The XGBoost makes important improvements to the GBDT algorithm in several ways, the most important of which is the improvement of the loss function of the GBDT algorithm. XGBoost has a number of advantages over GBDT, such as a second-order Taylor expansion of the loss function, the addition of a regular term, reduced model complexity, etc.

eXtreme Gradient Boosting uses a heuristic algorithm to optimize the objective function in a step-by-step sequence. Figure 1 illustrates the framework of general flow framework for optimizing the XGBoost mathematical model. During this process, a new function is added to the new mathematical model each time based on the original mathematical model, and the optimal result is gradually achieved by iterations.

In the following, we describe in detail the variables and parameters involved in Figure 1.

The mathematical model of XGBoost is shown in Eq (17) below. $({x}_{i}, {y}_{i})$ is the constituent element in the specified dataset $E = \left\{\right({x}_{i}, {y}_{i}\left)\right\}$. $f\left(x\right)$ represents a specific cart tree model. $\mathrm{\Gamma }$ is the hypothesis space consisting of all base classifiers Cart Tree (Classification), defined as shown in Eq (18) below.

The following Eq (19) represents the objective function of the model waiting to be optimized, which has two components. The first part represents the training error of the model. The second part is the regularization term, representing the sum of the complexity of all the trees. The definition of the regularization term of the objective function is shown in Eq (20) below.

The following second-order Taylor expansion transformation is applied to the error function. In Eq (21), the prediction of the model after the tth iteration has two components, the previous (t-1) model prediction and the prediction of the t-tree. The objective function can be rewritten as the following Eq (22) below.

The model needs to calculate the learning is the tree ${f}_{t}$. Changing Eq (21) into (23), ${g}_{i}$ and ${h}_{i}$ are defined in Eq (24) below.

Removing the constant term from Eq (23) gives Eq (25) below.

Take the content of the following Eqs (26) and (27) into the (25) objective function

The following equation is obtained.

Defining the set of samples on each leaf node j as ${I}_{j} = \{i\mid q({x}_{i}) = j\}$. The objective function (28) can be rewritten in the cumulative form of the leaf nodes as Eqs (29) and (30) below.

If q(x) has been determined (the structure of the tree has been determined), then the most predictive score for each leaf node can be found. The objective function is later minimised by making its derivative zero, as defined in Eq (31) below.

After substitution into the objective function, the final optimization result can be obtained, as defined in Eq (32) below.

4.   Our approach

4.1.   Overview of the framework

The XGB-SAD attack detection model can be seen as a four-stage process. Figure 2 illustrates the basic framework of the ensemble detection method XGB-SAD. Module 1 is the stage where the fake user is generated using the relevant parameters and the shilling attack model. The normal users are mixed with fake users into a rating matrix which is used as input for subsequent modules. Module 2 is the stage where the analysis rating matrix is processed using a double-view. We analyzed user rating time and the prevalence of user rating items, based on which we obtained the TPUS and UFM (user feature matrix). Module 3 is the stage where the set of base classifiers is generated and the model is constructed. The mathematical principle of gradient boosting is used to complete the iterative optimization of the objective function and loss function. Module 4 is the stage where the classification of malicious fake users is completed using the integrated strong classifier.

4.2.   Double-view data pre-processing module

In Module 1, the parameters related to the shilling attack are as follows: attack size, fill size, and attack model. In this stage, we do some basic processing of the dataset in preparation for the subsequent modules. We first took some of the users from the original dataset as the data for the subsequent training set. Table 3 lists the parameters used in shilling attack models. The dataset was attacked using the attack model in Table 3 to generate the dataset required for the experiment. The set of fake users and normal users are mixed into one rating matrix. Finally, this rating matrix is used as input to subsequent modules.

In Module 2, we will analyse and process the rating matrix. Table 4 lists the TPUS-PRE algorithm used in data pre-processing. In order to solve the problem that past methods mostly used a single angle to analyse the rating matrix, we combined the double-view perspective in the TPUS-PRE algorithm. The underlying idea of the TPUS-PRE algorithm is to use double-view to analyse the rating matrix.

The TPUS-PRE algorithm has four inputs, TRS, I, U and N. In Sections 3.1 and 3.2, we have analysed and defined TPUS collection based on the user's time and the user rating item. The TPUS-PRE algorithm calculates benchmark definitions A, B and C in the training set based on Eqs (8)–(10), followed by benchmark definitions D, E and F based on Eqs (12)–(14). Construct a matrix ${V}_{t}$ consisting of TPUS1 and a matrix ${V}_{b}$ consisting of TPUS2 and TPUS3. Then, combine ${V}_{b}$ and ${V}_{t}$ and assign the value to ${V}_{u}$. The final output of the algorithm is the UFM (user feature matrix). The role of the TPUS-PRE algorithm in the detection framework is to perform further processing of the rating matrix, which is provided as output to subsequent modules.

4.3.   Constructing detection mathematical model module

In Module 3, we will use the mathematical principles of eXtreme gradient boosting to construct the set of base classifiers. Table 5 lists the algorithms that use the training-set to generate the base classifier (named GCTS). The underlying idea of the GCTS algorithm uses the mathematical principle of extreme gradient boosting to construct each subtree and base classifier set. To address the problems that existed in past detection methods that mostly used a single classifier to handle attack detection, this paper considers the use of a collection of base classifiers in CGTS. We divide the training set into k subsets and use this as the basis for generating k base classifiers (Cart tree). The GCTS algorithm has four inputs, including number of iterations, train-set, base classifier cart tree and type label set. The final output of the algorithm is the base classifier set (Cart Tree Set).

After the start of the GCTS algorithm, we first use the TPUS-PRE algorithm in Table 4, with TRS as input. Then, we initialize the CTS set to the empty set. The mathematical model for XGBoost is given in Eqs (17)–(32) in Section 3.3, along with how to optimize the mathematical model and the loss function. The objective function is optimized using Eq (32) in Section 3.3. In each iteration, the objective function is first calculated according to Eq (19). Calculate ${g}_{i}$ and ${h}_{i}$ as parameter variables in the subsequent optimization process. Calculate ${f}_{i}\left(x\right)$ using Eq (26). After minimizing the objective function and loss function, ${f}_{i}\left(x\right)$ is incorporated into the CTS set. Execute the for loop repeatedly until N is reached. The result of the GCTS algorithm is a CTS. The role of the GCTS algorithm in the overall detection framework is to construct the set of base classifiers and prepare the predictions of subsequent base classifiers for combination.

4.4.   Malicious shilling attack detection module

In Module 4, we use the TPUS-PRE and the GCTS to construct the attack detection algorithm. Table 6 lists the algorithms used for shilling attack detection (called XGB-SAD). Two disadvantages of some past methods are that they only analyze the rating matrix from a single perspective of the user's rating value, ignoring other perspectives about the user. The other is that some methods use only a single classifier to handle the classification of malicious attackers. In the meantime, we have introduced the TPUS-PRE and GCTS algorithms. So based on the above foundation and in order to deal with the above problem in a reasonable way, the underlying idea of the XGB-SAD algorithm is to use UFM and CTS to combine the predictions of all the base classifiers to obtain a strong learner.

There are four inputs to the XGB-SAD, namely TES, CTS, TLS and THR. First, the algorithm takes the TES as input, calls the TPUS-PRE algorithm in Table 4, gets the output and assigns it to ${\mathrm{V}}_{\mathrm{u}}$. In a double loop, initializing threshold THR and using ${\mathrm{T}\mathrm{E}\mathrm{V}}_{\mathrm{u}}$ for the prediction of classifier ${f}_{k}$. After the predicted results${pre}_{{CTS}^{i}}\left(u\right)$ are obtained, the predicted for each user $p\left(u\right)$ are combined. At this point, the inner for loop is ended. Combining the prediction results and threshold THR of the base classifier (cart tree) for user u, the category $TLS\left(u\right)$ of user u is determined. At this point, the outer for loop is terminated. Merge $TLS\left(u\right)$ with ${TLS}_{r}$ and assign the merged result to ULS. The final output of the XGB-SAD algorithm is user label set (ULS). At the end of the XGB-SAD algorithm, the set of categories of normal user and fake user in Figure 2 is obtained.

5.   Experimental evaluation

5.1.   Experimental setup

This subsection first describes the pre-experimental preparations, including the experimental dataset, attack size, filler size, attack model and comparison algorithms. The proposed method can be used for datasets (Netflix, Amazon, MovieLens, etc.) with various numbers of users and data sparsity. For the purpose of our study, the following dataset were extracted from Movielens-100k ^[21]. The Movielens-100k contains detailed rating information for 1682 items from 943 users. The user's rating values for items range from "1" to "5", with "0" representing no rating, "1" representing the lowest preference for items and "5" representing the highest preference for items. Table 7 illustrates the user-item rating matrix after simple processing of the Movielens-100k dataset.

The key parameters used in the experiments are presented in the following Eqs (33) and (34). In Eq (33), N refers to the quantity and the ratio of ${N}_{attckuser}$ to ${N}_{u}$ in the system database refers to the attack size. In Eq (34), N refers to the quantity. In an attack profile, the ratio of ${N}_{IF}$ to ${N}_{item}$ in the system refers to the filler size.

The three comparison methods used in the experiments were PCA ^[16], Semi ^[22], and BAY ^[23]. PCA-SAD is a method that uses the unsupervised learning method PCA-SelectUsers to detect malicious fake users. Semi-SAD is a semi-supervised learning method. BAY-SAD combines multiple sets of base classifiers and uses the combined output to detect shilling attack.

5.2.   Evaluation metrics

This section lists several evaluation metrics used in this experiment to evaluate the performance of methods. Precision Eq (35) indicates the percentage of the sample with positive predicted outcomes that also had positive true cases. Recall Eq (36) indicates the percentage of all samples with positive true cases that also have a positive predicted outcome. F1-Measure Eq (37) is a weighted average of precision and recall, reflecting the comprehensive effect of the model.

5.3.   Experimental results and discussion

5.3.1.   Performance of three attack models

This subsection tests the evaluation metrics of the XGB-SAD under the attack models with the experimental parameters of filler size (3, 7, 10 and 15%) and attack size (3, 5, 7, 10, 12 and 15%). Figures 3–5 show the performance of our method for three attack models with the filler size fixed at 10%.

The Figure 3 below illustrates an overview of the variation in precision values. What can be clearly seen in this figure is that the precision value of the algorithm keeps improving and remains above approximately 0.9. When the attack size is small, the precision values under bandwagon attack are relatively low compared to random and average attack. As the experiments continued to increase the attack size, the precision of the algorithm gradually improves and remains at a high level.

The Figure 4 below illustrates an overview of the variation in recall values. When the size of the attack is small, the recall value of the algorithm under the random attack model is relatively low. The detection of the method under the random attack model is relatively poor compared to the other two models.

The Figure 5 below shows the trend of F1 values under three attack model. What can be clearly seen in this figure is that the F1 value of the method keeps improving and remains above approximately 0.94. When the attack size is small, the F1-measure under the bandwagon attack model is relatively low. As the value of the attack size increases, the F1-Measure under the three attack models gradually increase and reach the same value. The F1 value represents the comprehensive detection effect of a model. So based on the experiments in Figure 5, we can conclude the following. The comprehensive detection effectiveness of the algorithm is better under average attack.

In the above section, several sets of experiments were carried out and the results were summarized and analyzed. These experiments confirm the relatively good detection performance of our method even when the size of the attack is small. At the same time, the overall detection effectiveness of our approach can be maintained at a relatively high level in the face of three different attack models.

5.3.2.   Detection performance of comparative methods

This section tests the performance of several comparison algorithms under the attack models with the experimental parameters of filler size (3, 7, 10 and 15%) and attack size (3, 5, 7, 10, 12 and 15%). For a clear analysis of the experimental results, we have divided the following figures of the experimental data into two parts with filler sizes fixed at 3 and 10%. Figures 6–8 show one set of comparative experiments and Figures 9–11 show another.

1) When the fill size is fixed at 3%, the experimental results are as follows.

Figure 6 illustrates the F1 values for several methods under the random attack model. When the size of the attack is small, the F1 value of the XGB-SAD method is higher than the other methods, which indicates that the detection effectiveness of our method is better at this time. With the increase in attack size, the curve of BAY-SAD shows a small fluctuation, while the F1 values of all other methods are steadily increasing. It can be seen that the XGB-SAD method is more effective in detecting random attack.

The Figure 7 below illustrates the F1 values for several methods under average attack model. When the size of the attack is small, the F1 value of the XGB-SAD method is higher than the other methods. However, the F1 value of the BAY-SAD method is lower, which indicates that the BAY-SAD method is less effective in attack detection at this time under the average attack. With the increase in attack size, the curve of BAY-ASD and Semi-SAD show small fluctuations, while the F1 values of the other methods gradually increase converging to the same. As can be seen from the Figure 7, our approach is more effective in terms of the size of small-scale attacks and overall detection effectiveness.

Figure 8 illustrates the F1 values for several methods under bandwagon attack model. When the size of the attack is small, the F1 are lower for the BAY-SAD method. From the graph above we can see that the bandwagon attack model has added selected items compared to the above two basic attack models, which affects the detection performance of the method.

2) When the filler size is fixed at 10%, the experimental results are as follows.

The Figure 9 below illustrates the F1 values for several methods under the random attack model. When the size of the attack is small, the F1 value of the XGB-SAD method is higher than the other methods. With the increase in attack size, the curve of BAY-SAD shows a small fluctuation, while the F1 values of all other methods are steadily increasing. It can be seen that the XGB-SAD method is more effective in terms of the size of small attack size and overall detection effectiveness.

Figure 10 illustrates the F1 values for several methods under the average attack. When the size of the attack is small, the F1 values are higher for the XGB-SAD method and lower for the BAY-SAD method. With the increase in attack size, the curve of Semi-SAD shows small fluctuations, while the F1 values of the other methods gradually increase converging to the same. From the figure below we can see that the XGB-SAD method is more effective in detecting average attack.

The Figure 11 shows the F1 values for Comparison algorithms under bandwagon attack model. When the size of the attack is small, the F1 value of the XGB-SAD method is higher than the other methods. With the increase in attack size, the curve of BAY-SAD shows small fluctuations. As shown in figure below, the XGB-SAD method is more effective in detecting bandwagon attack.

6.   Conclusions

To counter the security threat posed by the shilling attack, many detection methods are proposed. The XGB-SAD improves on the limitations of past methods that mainly used a single view and a single classifier to analyze the user rating values. Our method analyzes the rating matrix using a doubleview of user rating time and user rating item, which in turn defines the TPUS collection. Then we perform heuristic iterative optimization of the model's objective function using eXtreme gradient boosting and integrate multiple sets of base classifiers into a strong classifier using the idea of ensemble learning. Finally, the strong classifier generated in the previous stage are used to complete the identification and detection of malicious shilling attackers. The experimental results show that XGB-SAD outperforms the comparison methods in terms of overall detection and detection precision and at small attack size.

In the future, our research has two directions, the first of which is the combination of external variables and high-performance classifiers with different settings base classifiers in our approach. Another is that although our method can theoretically be extended to other datasets, balancing the sparsity of different datasets and improving the generalizability of the method in the face of sophisticated attack models remains a key concern.

Acknowledgments

This work is supported by the University Nursing Program for Young Scholars with Creative Talents in Heilongjiang Province (No. UNPYSCT-2018208).

Conflict of interest

The authors declared that they have no conflicts of interest to this work. We declared that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.