Redis5 BloomFilter installation under mac and how to use it with python

Installation and use of Bloom filter

Installation and use of Bloom filter (BloomFilter) on Redis 5.x on Centos7

1 进入redis安装目录：cd /usr/local/redis-5.0.4
2. 下载插件： git clone https://github.com/RedisBloom/RedisBloom.git  
	# https://github.com/RedisBloom/RedisBloom 如果慢 可以使用外网访问
3. 进入插件目录： cd redisbloom/  （重命名之前为RedisBloom）
4. 执行： make
5. 修改 redis.conf，增加配置： loadmodule /usr/local/redis-5.0.4/redisbloom/redisbloom.so
6. 启动redis:  src/redis-server ./redis.conf
7. 连接客户端： src/redis-cli -p 6379 
8. 测试，先后执行： bf.add users francis     bf.exists users francis  
9. 更多内容可参考： https://oss.redislabs.com/redisbloom/

Usage of python
1. The first type Method to connect to redis Use native statements

from redis import StrictRedis
from django.conf import settings


class BfRedis:
    def __init__(self, db, host=settings.BF_REDIS_HOST, port=settings.BF_REDIS_PORT, password=settings.BF_REDIS_PASSWORD):
        self.client = StrictRedis(db=db, host=host, port=port, password=password)

    def bf_init(self, key: str, error_rate: float(), size: int):
        res = self.client.execute_command('BF.RESERVE', key, error_rate, size)
        return res

    def bf_exists(self, key, value):
        res = self.client.execute_command('BF.exists', key, value)
        return res

    def bf_add(self, key, value):
        return self.client.execute_command('BF.add', key, value)

    def bf_local_init(self, task_id, error_rate=0.0001, size=10000):
        """
        """
        key = f'bf_{task_id}'
        if self.client.exists(key):
            return True
        res = self.bf_init(key, error_rate, size)
        return res

    def bf_local_add(self, task_id, value):
        key = f'bf_{task_id}'
        res = self.bf_add(key, value)
        return res

    def bf_local_exists(self, task_id, value):
        key = f'bf_{task_id}'
        res = self.bf_exists(key, value)
        return res

    def bf_local_del(self, task_id):
        key = f'bf_{task_id}'
        res = self.client.delete(key)
        return res
# bf_redis = CrawlRedisClient(0)

1. Use python tool module

python2安装：pip install pybloom
python3安装：pip install pybloom-live

demo

from pybloom import BloomFilter, ScalableBloomFilter
bf = BloomFilter(capacity=10000, error_rate=0.001)
bf.add('test')
print 'test' in bf
sbf = ScalableBloomFilter(mode=ScalableBloomFilter.SMALL_SET_GROWTH)
sbf.add('dddd')
print 'ddd' in sbf

BloomFilter is a constant capacity filter, error_rate means that the maximum false positive rate is 0.1%, and ScalableBloomFilter is a variable capacity Bloom filter , it can continuously add elements. add The method is to add an element. If the element is already in the bloom filter, it returns true. If it is not, it returns fasle and adds the element to the filter. To determine whether an element is in the filter, just use the in operator.

The above is the detailed content of Redis5 BloomFilter installation under mac and how to use it with python. For more information, please follow other related articles on the PHP Chinese website!