Detailed explanation of collaborative filtering recommendation algorithm implemented in Python

Different data and different programmers write different collaborative filtering recommendation algorithms, but their core is the same. This article mainly introduces the complete code example of implementing collaborative filtering recommendation algorithm in Python, which has certain reference value. Friends who need it can For reference. Hope it helps everyone.

Test data

http://grouplens.org/datasets/movielens/

##The collaborative filtering recommendation algorithm is mainly divided into For:

1, based on users. Based on neighboring users, predict the uninvolved items that the current user has no preference, and calculate a sorted list of items for recommendation

2. Based on items. If users who like item A also like item C, then we can know that item A and item C are very similar, and user C likes item A, then we can infer that user C may also like item C.

Different data and different programmers write different collaborative filtering recommendation algorithms, but their core is the same:

1. Collect user preferences

1) Grouping different behaviors

2) Weighting different groups to calculate the total preferences of users

3) Data denoising and normalization

2. Find similar users (based on users) Or items (based on items)

3. Calculate the similarity and sort. Make recommendations for users based on similarity

The process of this example:

1. Initialize data

Get movies and ratings

Convert into data userDict to represent a certain A set of ratings for all movies of a user, and normalize the ratings by dividing them by 5

Convert to data ItemUser represents the set of all users who participated in rating a certain movie

2. Calculate all users Similarity with userId

Find out all users who watch movies that have intersection with userId

Loop and calculate the similarity with userId for these users

Get the relationship between user A and userId Union. The format is: {'Movie ID', [A user's rating, userId's rating]}, no rating is recorded as 0

Calculate the cosine distance between A user and userId, the larger the more similar it is

3. Generate a recommended movie list based on similarity

4. Output the recommended list and accuracy

#!/usr/bin/python3
# -*- coding: utf-8 -*-
from numpy import *
import time
from texttable import Texttable
class CF:
  def __init__(self, movies, ratings, k=5, n=10):
    self.movies = movies
    self.ratings = ratings
    # 邻居个数
    self.k = k
    # 推荐个数
    self.n = n
    # 用户对电影的评分
    # 数据格式{'UserID：用户ID':[(MovieID：电影ID,Rating：用户对电影的评星)]}
    self.userDict = {}
    # 对某电影评分的用户
    # 数据格式：{'MovieID：电影ID',[UserID：用户ID]}
    # {'1',[1,2,3..],...}
    self.ItemUser = {}
    # 邻居的信息
    self.neighbors = []
    # 推荐列表
    self.recommandList = []
    self.cost = 0.0

  # 基于用户的推荐
  # 根据对电影的评分计算用户之间的相似度
  def recommendByUser(self, userId):
    self.formatRate()
    # 推荐个数 等于 本身评分电影个数，用户计算准确率
    self.n = len(self.userDict[userId])
    self.getNearestNeighbor(userId)
    self.getrecommandList(userId)
    self.getPrecision(userId)

  # 获取推荐列表
  def getrecommandList(self, userId):
    self.recommandList = []
    # 建立推荐字典
    recommandDict = {}
    for neighbor in self.neighbors:
      movies = self.userDict[neighbor[1]]
      for movie in movies:
        if(movie[0] in recommandDict):
          recommandDict[movie[0]] += neighbor[0]
        else:
          recommandDict[movie[0]] = neighbor[0]

    # 建立推荐列表
    for key in recommandDict:
      self.recommandList.append([recommandDict[key], key])
    self.recommandList.sort(reverse=True)
    self.recommandList = self.recommandList[:self.n]

  # 将ratings转换为userDict和ItemUser
  def formatRate(self):
    self.userDict = {}
    self.ItemUser = {}
    for i in self.ratings:
      # 评分最高为5 除以5 进行数据归一化
      temp = (i[1], float(i[2]) / 5)
      # 计算userDict {'1':[(1,5),(2,5)...],'2':[...]...}
      if(i[0] in self.userDict):
        self.userDict[i[0]].append(temp)
      else:
        self.userDict[i[0]] = [temp]
      # 计算ItemUser {'1',[1,2,3..],...}
      if(i[1] in self.ItemUser):
        self.ItemUser[i[1]].append(i[0])
      else:
        self.ItemUser[i[1]] = [i[0]]

  # 找到某用户的相邻用户
  def getNearestNeighbor(self, userId):
    neighbors = []
    self.neighbors = []
    # 获取userId评分的电影都有那些用户也评过分
    for i in self.userDict[userId]:
      for j in self.ItemUser[i[0]]:
        if(j != userId and j not in neighbors):
          neighbors.append(j)
    # 计算这些用户与userId的相似度并排序
    for i in neighbors:
      dist = self.getCost(userId, i)
      self.neighbors.append([dist, i])
    # 排序默认是升序，reverse=True表示降序
    self.neighbors.sort(reverse=True)
    self.neighbors = self.neighbors[:self.k]

  # 格式化userDict数据
  def formatuserDict(self, userId, l):
    user = {}
    for i in self.userDict[userId]:
      user[i[0]] = [i[1], 0]
    for j in self.userDict[l]:
      if(j[0] not in user):
        user[j[0]] = [0, j[1]]
      else:
        user[j[0]][1] = j[1]
    return user

  # 计算余弦距离
  def getCost(self, userId, l):
    # 获取用户userId和l评分电影的并集
    # {'电影ID'：[userId的评分，l的评分]} 没有评分为0
    user = self.formatuserDict(userId, l)
    x = 0.0
    y = 0.0
    z = 0.0
    for k, v in user.items():
      x += float(v[0]) * float(v[0])
      y += float(v[1]) * float(v[1])
      z += float(v[0]) * float(v[1])
    if(z == 0.0):
      return 0
    return z / sqrt(x * y)

  # 推荐的准确率
  def getPrecision(self, userId):
    user = [i[0] for i in self.userDict[userId]]
    recommand = [i[1] for i in self.recommandList]
    count = 0.0
    if(len(user) >= len(recommand)):
      for i in recommand:
        if(i in user):
          count += 1.0
      self.cost = count / len(recommand)
    else:
      for i in user:
        if(i in recommand):
          count += 1.0
      self.cost = count / len(user)

  # 显示推荐列表
  def showTable(self):
    neighbors_id = [i[1] for i in self.neighbors]
    table = Texttable()
    table.set_deco(Texttable.HEADER)
    table.set_cols_dtype(["t", "t", "t", "t"])
    table.set_cols_align(["l", "l", "l", "l"])
    rows = []
    rows.append([u"movie ID", u"Name", u"release", u"from userID"])
    for item in self.recommandList:
      fromID = []
      for i in self.movies:
        if i[0] == item[1]:
          movie = i
          break
      for i in self.ItemUser[item[1]]:
        if i in neighbors_id:
          fromID.append(i)
      movie.append(fromID)
      rows.append(movie)
    table.add_rows(rows)
    print(table.draw())
# 获取数据
def readFile(filename):
  files = open(filename, "r", encoding="utf-8")
  # 如果读取不成功试一下
  # files = open(filename, "r", encoding="iso-8859-15")
  data = []
  for line in files.readlines():
    item = line.strip().split("::")
    data.append(item)
  return data

# -------------------------开始-------------------------------
start = time.clock()
movies = readFile("/home/hadoop/Python/CF/movies.dat")
ratings = readFile("/home/hadoop/Python/CF/ratings.dat")
demo = CF(movies, ratings, k=20)
demo.recommendByUser("100")
print("推荐列表为：")
demo.showTable()
print("处理的数据为%d条" % (len(demo.ratings)))
print("准确率： %.2f %%" % (demo.cost * 100))
end = time.clock()
print("耗费时间： %f s" % (end - start))

Summary

and above That’s all the content of this article about the complete code example of implementing collaborative filtering recommendation algorithm in python. I hope it will be helpful to everyone.

Related recommendations:

implementation of php+mysql collaborative filtering algorithm

Collaborative filtering

Tutorial on implementing collaborative filtering in Python

The above is the detailed content of Detailed explanation of collaborative filtering recommendation algorithm implemented in Python. For more information, please follow other related articles on the PHP Chinese website!