The wonderful use of bitmap in Redis

In Redis we often use set, get and other commands. If you are careful, have you noticed that there are several similar commands called setbit and getbit. What are they used for?

What is BitMap

It uses a bit to represent the value or status corresponding to an element, and the key is the corresponding element itself. We know that 8 bits can form a Byte, so the bitmap itself will greatly save storage space.

BitMap in Redis

Redis has added setbit, getbit, bitcount, etc. starting from version 2.2.0 Several bitmap related commands. Although it is a new command, no new data type is added, because commands such as setbit are just extensions of set.

setbit command introduction

InstructionSETBIT key offset value
ComplexityO(1)
Set or clear the value of the key (String) The bit value at offset (can only be 0 or 1).

Space occupation, and the time required to allocate space for the first time

On a 2010 MacBook Pro, the offset is 2^32-1 (512MB allocated) takes ~300ms, and the offset is 2^ 30-1 (allocated 128MB) takes ~80ms, offset 2^28-1 (allocated 32MB) takes ~30ms, offset 2^26-1 (allocated 8MB) takes 8ms.
The approximate space occupation calculation formula is: ($offset/8/1024/1024)MB

Usage scenario 1: User sign-in

Many websites provide a check-in function (data implementation is not considered here), and need to display the check-in status in the past month. How do we use bitmap? The code is revealed in one word!

<?php
$redis = new Redis();
$redis->connect('127.0.0.1');


//用户uid
$uid = 1;

//记录有uid的key
$cacheKey = sprintf("sign_%d", $uid);

//开始有签到功能的日期
$startDate = '2017-01-01';

//今天的日期
$todayDate = '2017-01-21';

//计算offset
$startTime = strtotime($startDate);
$todayTime = strtotime($todayDate);
$offset = floor(($todayTime - $startTime) / 86400);

echo "今天是第{$offset}天" . PHP_EOL;

//签到
//一年一个用户会占用多少空间呢？大约365/8=45.625个字节，好小，有木有被惊呆？
$redis->setBit($cacheKey, $offset, 1);

//查询签到情况
$bitStatus = $redis->getBit($cacheKey, $offset);
echo 1 == $bitStatus ? '今天已经签到啦' : '还没有签到呢';
echo PHP_EOL;

//计算总签到次数
echo $redis->bitCount($cacheKey) . PHP_EOL;

/**
* 计算某段时间内的签到次数
* 很不幸啊,bitCount虽然提供了start和end参数，但是这个说的是字符串的位置，而不是对应"位"的位置
* 幸运的是我们可以通过get命令将value取出来，自己解析。并且这个value不会太大，上面计算过一年一个用户只需要45个字节
* 给我们的网站定一个小目标，运行30年，那么一共需要1.31KB(就问你屌不屌？)
*/
//这是个错误的计算方式
echo $redis->bitCount($cacheKey, 0, 20) . PHP_EOL;

Usage scenario two: counting active users

Use time as the cacheKey, and then the user ID is offset. If it is active that day, set it to 1
Then how should I calculate certain days? As for the active users of /month/year (for the time being, it is agreed that only one day of online time within the statistical period is called active), please give the next redis command
CommandBITOP operation destkey key [key ...]
Description: Perform bit operations on one or more string keys that store binary bits, and save the results to destkey.
Note: The BITOP command supports any parameter of the four operations AND, OR, NOT, and /1024=6MB

Usage scenario three: User online status

I developed a project some time ago, and the other party provided me with an interface to query whether the current user is online. I don’t know how the other party does it, so I thought about it myself. Using bitmap is a space-saving and efficient method. It only requires one key, and then the user ID is offset. If it is online, it is set to 1, and if it is not online, it is set to 1. 0. Same as the above scenario, a 5000W user only needs 6MB of space.

//日期对应的活跃用户

$data = array(

'2017-01-10' => array(1,2,3,4,5,6,7,8,9,10),

'2017-01-11' => array(1,2,3,4,5,6,7,8),

'2017-01-12' => array(1,2,3,4,5,6),

'2017-01-13' => array(1,2,3,4),

'2017-01-14' => array(1,2)

);



//批量设置活跃状态

foreach($data as $date=>$uids) {

$cacheKey = sprintf("stat_%s", $date);

foreach($uids as $uid) {

$redis->setBit($cacheKey, $uid, 1);

}

}



$redis->bitOp('AND', 'stat', 'stat_2017-01-10', 'stat_2017-01-11', 'stat_2017-01-12') . PHP_EOL;

//总活跃用户：6

echo "总活跃用户：" . $redis->bitCount('stat') . PHP_EOL;



$redis->bitOp('AND', 'stat1', 'stat_2017-01-10', 'stat_2017-01-11', 'stat_2017-01-14') . PHP_EOL;

//总活跃用户：2

echo "总活跃用户：" . $redis->bitCount('stat1') . PHP_EOL;



$redis->bitOp('AND', 'stat2', 'stat_2017-01-10', 'stat_2017-01-11') . PHP_EOL;

//总活跃用户：8

echo "总活跃用户：" . $redis->bitCount('stat2') . PHP_EOL;

In fact, BitMap can be used in many scenarios (of course it will be subject to some restrictions), and ideas can continue to spread~ Friends are welcome to leave me a message to discuss~

In Redis we Commands such as set and get are often used. If you are careful, have you noticed that there are several similar commands called setbit and getbit. What are they used for?

What is BitMap

It uses a bit to represent the value or status corresponding to an element, and the key is the corresponding element itself. We know that 8 bits can form a Byte, so the bitmap itself will greatly save storage space.

BitMap in Redis

Redis has added

setbit

, getbit, bitcount, etc. starting from version 2.2.0 Several bitmap related commands. Although it is a new command, no new data type is added, because commands such as setbit are just extensions of set. setbit command introduction

Instruction

SETBIT key offset value

ComplexityO(1)
Set or clear the value of the key (String) The bit value at offset (can only be 0 or 1).
Space occupation, and the time required to allocate space for the first time

On a 2010 MacBook Pro, the offset is 2^32-1 (512MB allocated) takes ~300ms, and the offset is 2^ 30-1 (allocated 128MB) takes ~80ms, offset 2^28-1 (allocated 32MB) takes ~30ms, offset 2^26-1 (allocated 8MB) takes 8ms.

The approximate space occupation calculation formula is:

($offset/8/1024/1024)MB
Usage scenario 1: User sign-in

Many websites provide a check-in function (data implementation is not considered here), and need to display the check-in status in the past month. What should we do if bitmap is used? The code is revealed in one word!

<?php
$redis = new Redis();
$redis->connect('127.0.0.1');


//用户uid
$uid = 1;

//记录有uid的key
$cacheKey = sprintf("sign_%d", $uid);

//开始有签到功能的日期
$startDate = '2017-01-01';

//今天的日期
$todayDate = '2017-01-21';

//计算offset
$startTime = strtotime($startDate);
$todayTime = strtotime($todayDate);
$offset = floor(($todayTime - $startTime) / 86400);

echo "今天是第{$offset}天" . PHP_EOL;

//签到
//一年一个用户会占用多少空间呢？大约365/8=45.625个字节，好小，有木有被惊呆？
$redis->setBit($cacheKey, $offset, 1);

//查询签到情况
$bitStatus = $redis->getBit($cacheKey, $offset);
echo 1 == $bitStatus ? '今天已经签到啦' : '还没有签到呢';
echo PHP_EOL;

//计算总签到次数
echo $redis->bitCount($cacheKey) . PHP_EOL;

/**
* 计算某段时间内的签到次数
* 很不幸啊,bitCount虽然提供了start和end参数，但是这个说的是字符串的位置，而不是对应"位"的位置
* 幸运的是我们可以通过get命令将value取出来，自己解析。并且这个value不会太大，上面计算过一年一个用户只需要45个字节
* 给我们的网站定一个小目标，运行30年，那么一共需要1.31KB(就问你屌不屌？)
*/
//这是个错误的计算方式
echo $redis->bitCount($cacheKey, 0, 20) . PHP_EOL;

Usage scenario two: counting active users

Use time as cacheKey, and then the user ID is offset. If it is active on that day, set it to 1
Then how should I calculate certain days/months/ As for the active users in years (for the time being, it is agreed that online only one day within the statistical period is called active), please give the next redis command
CommandBITOP operation destkey key [key ...]
Description: Perform bit operations on one or more string keys that store binary bits, and save the results to destkey.
Note: The BITOP command supports any parameter of the four operations AND, OR, NOT, and /1024=6MB

Usage scenario three: User online status

I developed a project some time ago, and the other party provided me with an interface to query whether the current user is online. I don’t know how the other party does it, so I thought about it myself. Using bitmap is a space-saving and efficient method. It only requires one key, and then the user ID is offset. If it is online, it is set to 1, and if it is not online, it is set to 1. 0. Same as the above scenario, a 5000W user only needs 6MB of space.

//日期对应的活跃用户

$data = array(

'2017-01-10' => array(1,2,3,4,5,6,7,8,9,10),

'2017-01-11' => array(1,2,3,4,5,6,7,8),

'2017-01-12' => array(1,2,3,4,5,6),

'2017-01-13' => array(1,2,3,4),

'2017-01-14' => array(1,2)

);



//批量设置活跃状态

foreach($data as $date=>$uids) {

$cacheKey = sprintf("stat_%s", $date);

foreach($uids as $uid) {

$redis->setBit($cacheKey, $uid, 1);

}

}



$redis->bitOp('AND', 'stat', 'stat_2017-01-10', 'stat_2017-01-11', 'stat_2017-01-12') . PHP_EOL;

//总活跃用户：6

echo "总活跃用户：" . $redis->bitCount('stat') . PHP_EOL;



$redis->bitOp('AND', 'stat1', 'stat_2017-01-10', 'stat_2017-01-11', 'stat_2017-01-14') . PHP_EOL;

//总活跃用户：2

echo "总活跃用户：" . $redis->bitCount('stat1') . PHP_EOL;



$redis->bitOp('AND', 'stat2', 'stat_2017-01-10', 'stat_2017-01-11') . PHP_EOL;

//总活跃用户：8

echo "总活跃用户：" . $redis->bitCount('stat2') . PHP_EOL;

In fact, BitMap can be used in many scenarios (of course it will be subject to some restrictions), and ideas can continue to spread~ Friends are welcome to leave me a message to discuss~

More magical uses of bitmap in Redis For related articles, please pay attention to the PHP Chinese website!