Let's talk about the potential blocking points of AOF in Redis (summary)

What are the potential blocking points of AOF? The following article summarizes some potential blocking points of AOF in Redis. I hope it will be helpful to you!

What are the potential blocking points of AOF

1. When Redis uses the fork child process to rewrite the AOF file, there are potential Blocking risk

1), fork child process

Fork child process, the main thread will definitely be blocked at the moment of fork (note that fork will not be a one-time Copy all memory data to the child process), fork uses the Copy On Write mechanism provided by the operating system, in order to avoid the long-term blocking problem caused by copying a large amount of memory data at once to the child process. [Related recommendations: Redis video tutorial]

But the fork child process needs to copy the necessary data structures of the process, one of which is copying the memory page table (virtual memory and Physical memory mapping index table), this copy process will consume a lot of CPU resources. The entire process will be blocked before the copy is completed. The blocking time depends on the memory size of the entire instance. The larger the instance, the larger the memory page table. The longer the fork blocking time.

After the copy of the memory page table is completed, the child process and the parent process point to the same memory address space. That is to say, although the child process is generated at this time, it does not apply for the same memory size as the parent process.

When will the memory of the father and son processes be truly separated?

"Realistic copy", as the name suggests, actually copies the real data in the memory when writing occurs. During this process, the parent process may also be at risk of blocking, which is the scenario introduced below.

2), The parent process writes during AOF rewriting

The fork child process points to the same memory as the parent process address space, at this time the child process can perform AOF rewriting and write all the data in the memory to the AOF file.

But at this time, the parent process will still have traffic written. If the parent process operates an existing key, then at this time the parent process will actually copy the memory data corresponding to the key and apply for a new one. In this way, gradually, the memory data of the father and son processes begin to separate, and the father and son processes gradually have their own independent memory spaces. Because memory allocation is allocated in units of pages, the default is 4k. If the parent process is operating a bigkey at this time, it will take longer to reapply for a large block of memory, which may cause the risk of blocking.

In addition, if the operating system turns on the huge page memory mechanism (Huge Page, page size 2M), then the probability of blocking when the parent process applies for memory will be greatly increased, so in the Redis machine The Huge Page mechanism needs to be turned off. Every time Redis forks to generate RDB or AOF rewrite is completed, you can see in the Redis log how much memory space the parent process has reapplied for.

3) Why doesn’t AOF rewrite reuse AOF’s own logs?

AOF rewrite doesn’t reuse AOF’s own logs:

• One reason is that writing the same file between parent and child processes will inevitably cause competition problems. Controlling competition means that it will affect the performance of the parent process.

• The second is that if the AOF rewrite process fails, then the original AOF file is equivalent to being contaminated and cannot be restored and used. Therefore, Redis AOF rewrites a new file. If the rewriting fails, just delete the file directly. It will not affect the original AOF file. After the rewriting is completed, just replace the old file.

For more programming-related knowledge, please visit: Programming Video! !

The above is the detailed content of Let's talk about the potential blocking points of AOF in Redis (summary). For more information, please follow other related articles on the PHP Chinese website!