INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING & International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN

0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME TECHNOLOGY (IJCET)

ISSN 0976 6367(Print) ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), pp. 412-421 IAEME: www.iaeme.com/ijcet.asp Journal Impact Factor (2013): 6.1302 (Calculated by GISI) www.jifactor.com

IJCET
IAEME

RECOMMENDATION SYSTEM BASED ON ASSOCIATION RULES APPLIED TO CONSISTENT BEHAVIOR OVER TIME
Paulo J. G. Lisboa1, Huda Naji Nawaf2, Wesam S. Bhaya3 Liverpool John Moores University, 2University of Babylon, 3University of Babylon

ABSTRACT People's behavior or desires change with time so taking this factor into account may enhance the quality of the recommendations made by recommender systems. We propose a method to improve recommendation systems by taking into consideration changes in the behavior of users over time through the analyses of consistency in user behavior over two different time intervals. A sequence alignment tool is used to create two similarity networks of users, one for each time period. A single similarity network can be obtained by multiplying the two edge values for the similarity networks derived for the different time periods, enhancing the similarity between users whose behavior is consistently similar in both periods. Communities of users are then found using spectral algorithms and recommendations are made within each community with Association Rules modeling. The results are benchmarked against a comparable method when the two time intervals are merged into a single time period and when the association rules method is applied to single population model. The performance of predictions made with association rules is also compared against the performance of predictions with the Nave Bayes algorithm. The two algorithms are evaluated on the Hetrec2011dataset using standard accuracy metrics; e.g. recall and precision. Our experimental evaluation shows that the proposed association rules-based recommender system measuring consistency of behavior over time and segmenting the population of users into communities shows marginal but consistent improvements over standard personalization methods that do not take account of behavioral consistency over time and also compared with single population models. Moreover, the accuracy of recommendations with Association Rules consistency improves on Nave Bayes models in terms of accuracy and F-values. Keywords: Recommendation System, Association Rule, Sequence Alignment, Similarity Networks, Communities of Users. 1. INTRODUCTION

The initial development of recommender systems came as a result of widely used by wellknown commercial systems, (e.g. Amazon.com, Netflix.com), where these system try lure customers by offering to them what they prefer of products.
412

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME There are three approaches in recommendation system, collaborative filtering, content-based filtering, and hybrid approaches. Collaborative filtering recommends items to a user depending on taste of users neighborhood. Content-based systems recommend items based on item's neighborhood. Hybrid approaches exploit both content-based and collaborative filtering facilities [1]. Most researchers in collaborative filtering field have studied the behavior of users in order to develop these systems, because the success of these systems is heavily dependent on the knowledge of user behavior, which may change from one period to another. This change may be reflected on an affiliation of user to communities, so the user may belong to the community at a time and probably belongs to another community in another time, leading to a change in the structure of the network which in turn reflects on the quality of the recommendations. In this paper, we suggest a technique to improve the collaborative filtering. We have focused on enhancing recommendations by analyzing the behavior of users in two different time periods by modeling the behavior of each user as a sequence of actions chronologically arranged. Sequence alignment tool is used to alignment these sequence of users to create similarity networks. The key steps of this work are twofold: (i) First, we build similarity networks to model the stability over time of user behavior, (ii) we used the latest behaviors of users for evaluation purposes, and i.e. these behaviors are not subject to analysis. The evaluation of recommendation system is a very important step. The evaluation can tell the extent the system inform predictions about the real world. Most methodologies of recommendation system are tested by splitting dataset into training, and test set. In this paper, each sequence for a user is split into three periods; the last one is test part. Because each sequence is arranged chronologically, this part of sequence reflects the future behavior of user. So, in our work we try to predict the future behavior of user, leading to more reality testing. The reset of this paper is organized as follow. Section 2 presents dataset, and how extracts sequences of users from dataset. Section 3 describes the neighborhood models. In section 4, we discuss the association rules as prediction algorithm. Performance of our proposed system is presented in section 5. Experimental results are displayed in section 6, and were discussed in section 7. 2. DATASET AND EXTRACTION SEQUENCES OF USERS In our study, HetRec 2011 Data sets (MovieLens rating data with IMDb and Rotten Tomatoes links) has been used. The datasets were generated by the Information Retrieval Group at Autnoma University of Madrid (http://ir.ii.uam.es), it has 86,000 ratings from 2113 users. This dataset is associated with IMDB (http://www.imdb.com/genre/) to create 19 genres for movies. The data was collected between September of 1997 and December of 2009. The sequence for each user is extracted from dataset as follows: i. Collect only ids of movies and the time stamp associated with an individual user. ii. Arrange the movies chronologically according to the time stamp to create a sequence of movies viewing actions. iii. Divide the sequence into three periods: a- 1st period from 1-Sep-1997 to 1-Oct-2001 b- 2nd period from 2-Oct-2001 to 1-Nov.-2005. c- 3rd period from 2-Nov-2005 to 30-Dec-2009. iv. Map each id of movie to its genre according to the movie taxonomy. Figure 1 shows activities of users over time in used data set. As we noticed the activities are linearly increasing until 2007, where the users were less active in first period and most activity in third period (evaluation). The same curve we got in figure 2 that show the active users in each

413

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME period. We have to use only the users involved in all three periods, so the number of users is too small as long as the users have not have the same level of activity.

Figure 1: Activities of users over time

Figure 2: The active users in each period 3. NEIGHBORHOOD MODELS Neighborhood models are created depending on the similarity relationships among either users or items. We are interested to find out the neighborhood depending on the similarity of relationships among users. Finding out neighborhood requires two steps: i. Create similarity networks ii. Analysis these similarity networks to detect communities (neighborhoods) 3.1. Create Similarity Networks Sequence alignment tools can be used to build similarity networks of users. It is modified sequence comparison method originally applied in bioinformatics. If we can create sequences from the viewing of movies for all users, so we can compare these sequences in same way employed in bioinformatics. Any dataset can subject to following comparison [2]: 1- If we can placing the activities into meaningful collection, 2- If there is temporal dimension for each activity, 3- If there is a dimension to compare each activity with another to obtain a score of similarity.
414

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME Optimal Matching is a global alignment algorithm applied in social science field as to analyze non-biological sequences as simulation to Needleman-Wunsch algorithm that applied in bioinformatics field [3]. As mentioned in section 2, each sequence is split into three periods; the first two periods are subjected to analysis. So, optimal matching is applied on the two first periods of all sequences of users to create two similarity networks. By multiplying the two similarity networks, a single similarity network is obtained which shows users with similar behavior over both periods. Optimal Matching starts by aligning two sequences using three operations, namely replacement, insertion and deletion. A combined insertion and deletion combination is called indel [3]. For example, if we have two sequences, the best alignment for them is showing in figure 3:

Figure 3: The optimal alignment between two sequences. Vertical line indicate matching items in the two sequences So, the resultant alignment has 2 mismatches, 6 matches, and 4 gaps. In principle, a weight is assigned to each replacement, insertion or deletion [3]. These weights reflect the cost of making the alignment between sequences to their total. This is a measure of the dissimilarity between the two sequences. From our experiments, the best results have been gotten when set the following weights: Gap and mismatch=0, Match=2 So, the final score for this alignment is: (6*6)+(4*0)+(2*0)=36. Sometimes, the difference in length between sequences is problem. So, Abbott and others found a way to deal with this problem by dividing the final score of alignment by the length of the longer sequence of the pair [3]. In our work, we deal with problem by the final score of alignment by length of resultant alignment as K. Duraiswamy and V. ValliMayil [4]. There are some papers dealing also with sequences of actions in the fields of web traffic and ecommerce. In particular, P. Liu and L. Hai propose a Sequence Alignment Technique (SAT) collaborative filtering (CF) recommendation methodology using Product Taxonomy (SAT-PT), to address the sparsity and scalability problems of current CF based recommender systems. The author used a simple hierarchical clustering algorithm to form neighborhoods [5]. In contrast, A. Banerjee and J Ghosh used weighted longest common subsequences for clustering web users. The purpose of clustering users is finding group of similar users in their interest based on their clickstreams, which takes into account both the trajectory taken through a website and the time spent at each page. Paper [4] focuses on a method to find session similarity by sequence alignment using dynamic programming and proposes a model such as similarity matrix for representing session similarity measures [6]. 3.2. Detect Communities (Neighborhoods) The spectral clustering algorithm can be used to cluster data using the eigenvectors of a similarity/affinity matrix derived from a dataset.

415

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME The key step of algorithm is the calculation of a modularity score, by which decide if the community can be split into two communities or not. The good division of a network into communities, it is one in which there are fewer than expected edges between communities or if the number within groups is more significantly [7]. So, the modularity matrix calculates the different between the number of edges between a pair of nodes, Aij, and the expected number derived from the order (i.e. total number of edges) of each node, ki, kj when placed at random. The community structure is derived by a hierarchical process where the total set of nodes is partitioned into two groups at a time, depending on the following modularity score [7]:
1 1 4 2 4

For a partition to take place, this modularity score must be increased .The similarity network Aij that used here is result from the alignment of users sequences. Each element in matrix represents the score of similarity between i and j, ki And kj are the degrees of vertices i and j respectively. So, kikj /2m is the expected number of edges between i and j, and m is the total number of edges [7]. 4. RECOMMENDER USING SIMPLE ASSOCIATION RULES

Recommendations list for the customers in the same community is predicted using association rule learning as prediction algorithm. The problem of association rule mining is defined as in [8]: Let I i , i , . . i a set of binary attributes called items. Let D t , t , . , t be a set of transactions called the database. Each transaction in D has a unique transaction ID and contains a subset of the items in I. A rules defined as an implication of the form xy where x,y I and xy=. The sets of items x and y are called antecedent and consequent of the rule respectively. To select interesting rules from the set of all possible rules, constraints on various measures of significance and interest can be used. The bestknown constraints are minimum thresholds on support and confidence. The support supp(x) of an item set x is defined as the proportion of transactions in the data set which contain the item set. The confidence of a rule is defined confx y=suppx ysuppx. In this paper, we used single item in antecedent and consequent of the rule, so for each item in antecedent of a rule, the item in consequent is considered as recommendation. 5. PERFORMANCE MEASURES OF RECOMMENDATION ALGORITHM In the field of machine learning, a confusion matrix is a specific table layout that allows visualization of the performance of an algorithm [10]. Table (1) shows the observations that represent the source of real conditions, the real conditions in our case is the third period (evaluation period), against predicted recommendations that result from our model. Each row of the matrix represents the films that have been automatically predicted by association rules algorithm, while each column represents the actual films that have been seen in evaluation period.

416

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME Table 1. The confusion matrix compares the predictions with the observations. Observed Positive: film Negative: film viewed not viewed a b c d

Predicted Positive: film recommended Negative: film not recommended

Several standard terms have been defined for the two class matrix. The accuracy (AC) is the proportion of the total number of predictions that were correct. It is determined using the equation: . The recall or true positive rate (TP) is the proportion of positive cases that were correctly identified, as calculated using the equation: . Finally, precision (P) is the proportion of the predicted positive cases that were correct, as calculated using the equation: . . F-measure considers both precision and recall providing a single measurement for a system: . . . .

In order to give different weights to precision and recall, the F-measure was derived so that F measures the effectiveness of retrieval with respect to a user who attaches times as much weighted to recall as precision. The parameter generally take the values 0.5, 1 and 2, with higher values of placing a greater emphasis on getting the predictions right in the sense of achieving a higher positive predictive value. . . . . . F1 is the traditional F-measure or balanced F-score because recall and precision are evenly weighted. Values of < 1 emphasize precision, while values of > 1 emphasize recall [9]. 6. EXPERIMENTAL RESULTS In the following experiments, the proposed system based on association rules is benchmarked against a comparable method when the two time intervals are merged into a single time period and when the association rules method is applied to single population model. The performance of methods based on association rules is compared with proposed system in [11].

417

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME 6.1. First Experiment The predictive performance of association rules system is presented quantitatively and graphically in table2. The system recommended 3 and 5 top items for 196 users. Table2: F-Measure and accuracy for predictive performance of association rules
Methods Single population Model (N-top=5) (N-top=3) One period (communities) (N-top=5) (N-top=3) Two periods(communities)(N-top=5) (N-top=3) F1 62% 57% Personalized Model 61% 57% 63% 59% F2 69% 55% 68% 55% 70% 56% F0.5 57% 59% 56% 60% 57% 61% Accuracy 62% 65% 61% 66% 62% 67%

6.2. Second experiment The predictive performance of Nave Bayes system [11] is presented in this experiment for comparison the performance of association rules prediction algorithm with it. The system recommended 5 and 3 top items for only 134, and 192 users respectively. Table 3 shows the performance of Nave Bayes prediction system. Table3: F-Measure and accuracy for predictive performance of Nave Bayes.
Methods Single Population Model (N-top=5) (N-top=3) F1 F2 65% 54% 65% 56% 66% 56% F0.5 55% 59% 55% 61% 56% 61% Accuracy 60% 65% 60% 66% 61% 67%

59% 57% Personalized Model One Period (communities) (N-top=5) 59% (N-top=3) 58% Two Periods(communities) (N-top=5) 60% (N-top=3) 58%

6.3 Third experiment In this experiment, association rules prediction algorithm applied on same 134 users in second experiment when N-top=5. For N-top=3, the two algorithms recommended the same 192 users. Table 4 shows the performance of association rules algorithm on 134, and 192 users. Figure 4, 5 and 6 show the comparison between Nave Bayes algorithm and association rules algorithm in terms of single population model, personalized model (one period), and personalized model (two periods) when N-top=5. Table 4. F-Measure and accuracy for predictive performance of association rules
Methods F1 Single population Model (N-top=5) 61% (N-top=3) 57% Personalized Model One period (communities) (N-top=5) 63% (N-top=3) 57% Two periods(communities) (N-top=5) 64% (N-top=3) 58% F2 67% 54% 69% 55% 70% 56% F0.5 56% 59% 58% 59% 58% 61% Accuracy 61% 65% 62% 66% 64% 67%

418

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME

Figure 4: Comparison between the two algorithms in terms of non-personalized model

Figure 5: Comparison between the two algorithms in terms of personalized model (one period)

Figure 6: Comparison between the two algorithms in terms of personalized model (two periods)

Figure 7: Precision-Recall curve of personalized model (two periods) using association-rules, and Nave-Bayes algorithms

7. DISCUSSION AND CONCLUSION A total 205 users who have actions in both periods in the modeling data behavior and a nonempty set of films viewed during the evaluation period. These users have been subjected to the study The system in all experiments remove some users because of they either have no new items in the evaluation period or have less than the determined number of items (N) in recommendation list. The first experiment was about the performance of Association rules as prediction algorithm on 205 users, where the system recommended fixed length (N) of items. It recommended 5 and 3 items for 196 users. Table 2 shows the comparison of performance of our algorithm against single population model, and personalized model (one period).The performance of single population algorithm (baseline) is outperformed by other two systems as expected when N=3, and our system outperformed the personalized model (one period) .When N=5, single population is still less
419

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME performance from proposed system in terms of F1 and F2, but superior personalized model (one period) in terms of F-values and accuracy. In second experiment, we applied Nave Bayes based recommendation system with the same dataset. The system recommended 5 items for only 134 users, and 3 items for 192 users. As we noticed, the performance of single population model has lower rank when N=3 as expected, while the single population is in line with personalized- (one period) when N=5. Again, the proposed system is slightly outperforming the both when N=3 and N=5. The two algorithms works on the same dataset, but the condition of each algorithm may impose a different number of users. Thus, we succumb the two algorithms the terms of the algorithm that accept the least number of users to make the comparison between their performance more accurate. So, the association rules based recommendation system has been applied on the same 134 and 192 users when N=5and N=3 respectively as described in experiment 3. Tables 3 and 4 tell some interesting things. On the one hand, we will compare the performance of Association Rules and Nave Bayes at first. As we noticed when N=3, the performance of the two algorithms are comparable in terms of F-values and accuracy in case of single population and personalized- (two periods). In case of personalized (one period), the performance of Nave Bayes is few better than Association Rules. The performance of the two algorithms is more different when N=5. The performance of Association Rules is significantly outperformed that of Nave Bayes in case of single population, personalized (one period and two periods). On other hand, we can compare the performance of the both personalized models; one period and two periods when N=3. As shown in table 3, the two methods are comparable except the accuracy of two periods is bet better than that of one period. Whereas in table 4, the personalized(two periods) model slightly outperformed the one period in terms of F-values and accuracy. The same comparison has been done when N=5. The performance of personalized- (two periods) is better than that of personalized-(one period) in both algorithms (tables 3 and 4). In general, compared to the performance of the algorithm in the case of existence communities, and in the case of there are not show the performance of single population has lower rank than that of communities (personalized models) in both algorithms, and whatever value of N. Worth mentioned, recommendation system based on association rules is more sensitive for existing communities than that of Nave Bayes. Figure 4, 5 and 6 report the performance of the algorithms in case of associative rules and Nave Bayes. Generally, the performance of Associative Rules in case of personalized-(two periods) model is best as shown in precision-recall curve ( see figure 7). Improve performance over Nave Bayes by 2% to 4% for F-values. The two algorithms did not recommend items for significant number of users when N=8, and N=10. 8. REFERENCES [1] O. Kirmemis, and A. Birturk: A Content-Based User Model Generation and Optimization Approach for Movie Recommendation, 6th Workshop on Intelligent Techniques for Web Personalization & Recommender Systems, AAAI Press, 2008. A. Godwin Time Web: Comparing University-Spaced Time Sequences Using Social Network Analysis of Local Alignment Pairs, M.S. Thesis, University of North Carolina at Charlotte, 2008. A. Abbott and A. Tsay, "Sequence analysis and optimal matching methods in sociology," Journal: Sociological Methods & Research, vol. 29, no. 1, pp. 3-33, 2000.
420

[2]

[3]

International Journal of Computer Engineering and Technology (IJCET), ISSN 09766367(Print), ISSN 0976 6375(Online) Volume 4, Issue 4, July-August (2013), IAEME [4] K. Duraiswamy, V. Mayil, "Similarity matrix based session clustering by sequence alignment using dynamic programming," Journal of Computer and information Science," 1(3): pp. 66- 72, 2008. [5] P. Liu, L. Hai, "Application of sequence alignment technique to collaborative recommendations in e-commerce," 2010 International Conference on E-Product E-Service and E-Entertainment (ICEEE), vol. 1, pp. 1-3, 2010 [6] A. Banerjee ,J Ghosh, "Click stream clustering using weighted longest common subsequences," In Proceedings of the Web Mining Workshop at the 1st SIAM Conference on Data Mining. pp. 33-40, 2001. [7] M. Newman, Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103, 8577-8582 (2006) [8] A. Geyer-Schulz, M. Hahsler, Evaluation of Recommender Algorithms for an Internet Information Broker based on Simple Association Rules and on the Repeat-Buying Theory. Proceedings of fourth webkdd workshop: Web Mining for Usage Patterns & User Profiles. pp. 100-114, 2002. [9] C. Manning, P. Raghavan, H. Schtze, An Introduction to Information Retrieval, Cambridge University Press, 2008. [10] VA. Narayana, P. Premchand, A. Govardhan , Performance and Comparative Analysis of the Two Contrary Approaches for Detecting Near Duplicate Web Documents in Web Crawling . International Journal of Electrical and Computer Engineering (IJECE), Vol. 2, No. 6, pp. 819~830, 2012 [11] P. Lisboa, H. Nawaf, W. Bhaya, Improving Recommendation Systems by Modeling the Stability of Implicit Behaviour. The Post Graduate Network Symposium (PGNet2013), Liverpool, UK, 2013 [12] Madhubala Myneni, Dr.M.Seetha, Feature Integration for Image Information Retrieval Using Image Mining Techniques International Journal of Computer Engineering & Technology (IJCET), Volume 3, Issue 3, 2012, pp. 273 - 281, ISSN Print: 0976 6367, ISSN Online: 0976 6375

421