【redis存储结构设计】存储坐标点及其多维度点击数

Question

我现在有个需求 需要记录页面点击数据，上游吐到redis中， 上游怎么吐到redis中对我们来说是透明的， 我们只用关心redis中如何存储就好。 查询需求： 查询某天某页面下所有点击数，即有效点击总数+无效点击总数 ...

PHPz · Answer

That is, when there are many keys, the query speed is very slow

Many of the key queries are slow. Does this mean that the zset actually clicked on the last query is used?

Not sure how many resolutions there will be? You can modify the key of zset not to have resolution, but to have resolution in value. This can reduce a lot of keys. If your search conditions have resolution, you can do some filtering after searching for value, and the speed should be very fast.

But the box selection behavior is because the range is variable
Frame selection query (equivalent to range query) Query on a certain day, a certain page, a certain resolution
The total number of valid clicks and the total number of invalid clicks at coordinate points in a certain range (such as 100

It’s like asking users to manually draw an area for search. Can you consider changing this condition to include the entire image? Cut into 10 parts (100 parts, 10,000 parts). Each part is a square. The condition can only select a certain square, rather than just drawing it randomly. In this way, the data in each square can be "summarized" predictably. .

Let’s talk about these first, see if it helps, if you still need to optimize, you can modify the query description in the question. There are some places that can be supplemented by your brain, but I don’t know if you want to express this, so I will give you a simpler one. Write the examples in detail and use typesetting, it looks very tiring

I wrote them separately. Here is what you have done after correcting the question

First of all, you are not using the essence of zset, which is automatically sorting the index according to scop. It seems that you must not understand the resolution I mentioned above when you put it in value. Let me give you an example

A zset records the data of a certain page and a certain resolution on a certain day
The key is date_pageid_resolution and the member is: valid OR invalid_browser_region
score is the number of clicks
Example: key : 20140908_0001_1000
member: 0_1_1 0 corresponds to invalid clicks, 1 corresponds to QQ browser in the browser table, 1 corresponds to Shanghai in the region table
score:10

Suppose there are 3 resolutions: A, B, C
Saving the key as you said will look like this
20140908_0001_A
20140908_0001_B
20140908_0001_C
The storage method I am talking about is
key:20140908_0001
member:valid OR invalid_browser_region_number of clicks
score:resolution

When searching like this, you actually only need to get the 0001 page of the day 20140908 (just 1 key), and then range A resolution and look at its members. This is not easy to use because it does not display the nice resolution. It's not interesting here. There are problems with using zset in this case.

The above is just an example! Actually, don't do this. There is a better way. After you revised the question and understood the requirements, I came up with a new approach.

zset：data set
key:date-page-resolution
score: coordinates (think about turning x and y into a number)
member: browser-region-number of valid clicks-number of invalid clicks

If the date becomes an optional range, this set is needed to specifically store the date. We call it: date set
key:page
score:date
member:data set key
The purpose of the date set is to index the data set key. Your method of using key() is very slow because it will perform an all search. Your example is a certain day. I understand that there may be no date range, so the date set can be unnecessary. Similarly, if there are too many resolutions and it is impossible to master, you can also imitate this set to make a collection of keys!

Then there are two coordinates zset. I didn’t look at them carefully. Let’s think carefully about using zset.

You gave 4 query examples below

A Query the number of all clicks on a certain page on a certain day, that is, the total number of valid clicks + the total number of invalid clicks

B Query the total number of valid clicks and the total number of invalid clicks on a certain page and a certain resolution on a certain day

C Query all the coordinate points and number of clicks on a certain page and a certain resolution on a certain day

D Box selection query (equivalent to range query) Query the total number of valid clicks and the total number of invalid clicks at a certain range (such as 100

A: You said there are 3 resolutions, then add 3 resolutions after the key, range 0 and -1 are all taken
20150415-page1-1380,20150415-page1-1190,20150415-page1-1000

B: This is good. Just check one key and get all range 0 and -1
20150415-page1-1380

C: Okay, you can also get the coordinates for the first two, but you don’t have a show

D: After using your coordinate set to get the key, check the data set range coordinates

I finished writing, but I found a small problem when checking for typos. It seems that you need to record the valid and invalid browsers in each region? If it is not necessary, the member in the data set can just record valid and invalid numbers. If it is necessary, the design needs to be considered based on the number of browsers in the region. Your question does not seem to introduce this aspect.

PHP中文网 · Answer

Maybe my understanding of redis is different from the questioner’s. According to my idea, to achieve the above requirements may be

Remember log, etl transfer data

Finally available for inquiry

查询需求：

需求解释：

当前实现

使用4个`zset`来实现需求。

当前实现存在的问题

查询需求：

需求解释：

当前实现

使用4个zset来实现需求。

当前实现存在的问题

使用4个`zset`来实现需求。