Table of Contents
AOF实现机制简介
配置文件redis.conf中有关AOF参数配置解释
appendonly
appendfsync
no-appendfsync-on-rewrite
appendfilename
redisServer数据结构中有关AOF的参数
initServerConfig函数对AOF参数的初始化及注释
小结
Home Database Mysql Tutorial Redis数据持久化机制AOF原理分析之配置详解

Redis数据持久化机制AOF原理分析之配置详解

Jun 07, 2016 pm 03:28 PM
aof redis analyze principle Endurance data mechanism Configuration

本文所引用的源码全部来自Redis2.8.2版本。 Redis的配置文件为redis.conf, 对配置文件的解析代码见config.c 原创文章转载请注明,本文来自http://blog.csdn.net/acceptedxukai/article/details/18135219 AOF实现机制简介 AOF 即Append Only File,实现机制:R

本文所引用的源码全部来自Redis2.8.2版本。

Redis的配置文件为redis.conf, 对配置文件的解析代码见config.c

原创文章转载请注明,本文来自http://blog.csdn.net/acceptedxukai/article/details/18135219

AOF实现机制简介

AOF 即Append Only File,实现机制:Redis将数据库做个快照,遍历所有数据库,将数据库中的数据还原为跟客户端发送来的指令的协议格式的字符串,然后Redis新建一个临时文件将这些快照数据保存,待快照程序结束后将临时文件名修改为正常的aof文件名,原有的文件则自动丢弃,由于在快照进行的过程中可能存在新增的命令修改了数据库中的数据,则在快照程序结束后需要将新修改的数据追加到aof文件中,后续的从客户端过来的命令都会不断根据不同的安全级别写到磁盘里面去。这样就支持了实时的持久化,只是可能会有短时间内的数据丢失,对一般系统还是可以容忍的。

配置文件redis.conf中有关AOF参数配置解释

appendonly

含义:Redis Server是否开启AOF持久化机制

在config.c函数loadServerConfigFromString解析代码为

 

1

2

3

4

5

6

7

if (!strcasecmp(argv[0],"appendonly") && argc == 2) {

    int yes;

    if ((yes = yesnotoi(argv[1])) == -1) {

        err = "argument must be 'yes' or 'no'"; goto loaderr;

    }

    server.aof_state = yes ? REDIS_AOF_ON : REDIS_AOF_OFF;

}

Copy after login

appendfsync

含义:Redis将OS数据缓冲区中数据刷新到磁盘的策略

# appendfsync always 只要有新添加的数据就fsync
appendfsync everysec 支持延迟fsync
# appendfsync no 不需要fsync
config.c中解析代码

1

2

3

4

5

6

7

8

9

10

11

12

if (!strcasecmp(argv[0],"appendfsync") && argc == 2) {//AOF fsync策略

    if (!strcasecmp(argv[1],"no")) {

        server.aof_fsync = AOF_FSYNC_NO;

    } else if (!strcasecmp(argv[1],"always")) {

        server.aof_fsync = AOF_FSYNC_ALWAYS;

    } else if (!strcasecmp(argv[1],"everysec")) {

        server.aof_fsync = AOF_FSYNC_EVERYSEC;

    } else {

        err = "argument must be 'no', 'always' or 'everysec'";

        goto loaderr;

    }

}

Copy after login

no-appendfsync-on-rewrite

含义:当rewrite AOF子进程或RDB子进程正在执行时,Server是否支持fsync,即当新修改的数据写入AOF文件后,是否将数据刷新到硬盘

config.c中解析代码

1

2

3

4

5

if (!strcasecmp(argv[0],"no-appendfsync-on-rewrite") && argc == 2) {

    if ((server.aof_no_fsync_on_rewrite= yesnotoi(argv[1])) == -1) {

        err = "argument must be 'yes' or 'no'"; goto loaderr;

    }

}

Copy after login
server.aof_no_fsync_on_rewrite控制该机制的实现代码在aof.c的flushAppendOnlyFile函数中,具体代码为

1

2

3

4

5

//如果不支持fsync,或者aof rdb子进程正在运行,那么直接返回,

//但是数据已经写到aof文件中,只是没有刷新到硬盘

if (server.aof_no_fsync_on_rewrite &&

    (server.aof_child_pid != -1 || server.rdb_child_pid != -1))

        return;

Copy after login

appendfilename

含义:AOF的文件名

auto-aof-rewrite-percentage 100

auto-aof-rewrite-min-size 64mb

这两个配置就暂不解释,见下面redisServer结构体中中有关AOF参数的注释

redisServer数据结构中有关AOF的参数

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

struct redisServer {

    …………

     

    /* AOF persistence */

    int aof_state;                  /* REDIS_AOF_(ON|OFF|WAIT_REWRITE) */

    int aof_fsync;                  /* Kind of fsync() policy */

    char *aof_filename;             /* Name of the AOF file */

    int aof_no_fsync_on_rewrite;    /* Don't fsync if a rewrite is in prog. */

    int aof_rewrite_perc;           /* Rewrite AOF if % growth is > M and... */

    off_t aof_rewrite_min_size;     /* the AOF file is at least N bytes. */ aof文件最小字节数

    off_t aof_rewrite_base_size;    /* AOF size on latest startup or rewrite. */ 上一次aof文件的大小

    off_t aof_current_size;         /* AOF current size. */ 当前aof文件的大小

    int aof_rewrite_scheduled;      /* Rewrite once BGSAVE terminates. */

    pid_t aof_child_pid;            /* PID if rewriting process */ 子进程的pid

    list *aof_rewrite_buf_blocks;   /* Hold changes during an AOF rewrite. */

    sds aof_buf;      /* AOF buffer, written before entering the event loop */

    int aof_fd;       /* File descriptor of currently selected AOF file */ aof文件描述符

    int aof_selected_db; /* Currently selected DB in AOF */

    time_t aof_flush_postponed_start; /* UNIX time of postponed AOF flush */

    time_t aof_last_fsync;            /* UNIX time of last fsync() */

    time_t aof_rewrite_time_last;   /* Time used by last AOF rewrite run. */

    time_t aof_rewrite_time_start;  /* Current AOF rewrite start time. */

    int aof_lastbgrewrite_status;   /* REDIS_OK or REDIS_ERR */

    unsigned long aof_delayed_fsync;  /* delayed AOF fsync() counter */

    int aof_rewrite_incremental_fsync;/* fsync incrementally while rewriting? */

     

    …………

}

Copy after login

字符串aof_buf与链表aof_rewrite_buf_blocks分别是追加AOF文件与rewrite AOF文件后处理差异数据的重要参数

initServerConfig函数对AOF参数的初始化及注释

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

server.aof_state = REDIS_AOF_OFF;            //AOF文件是否开启

server.aof_fsync = REDIS_DEFAULT_AOF_FSYNC;  //fsync策略,默认为每秒fsync

server.aof_no_fsync_on_rewrite = REDIS_DEFAULT_AOF_NO_FSYNC_ON_REWRITE;//aof rdb子进程运行中是否支持fsync,即写入AOF文件后,将数据刷新到硬盘

server.aof_rewrite_perc = REDIS_AOF_REWRITE_PERC; //自动rewrite增量值

server.aof_rewrite_min_size = REDIS_AOF_REWRITE_MIN_SIZE; //AOF文件最小字节数

server.aof_rewrite_base_size = 0;   //自动rewrite计算aof文件增量的基数,等于上一次aof文件的字节数

server.aof_rewrite_scheduled = 0;   //rewrite任务计划,当客户端发送bgrewriteaof指令,如果当前rewrite子进程正在执行,那么将客户端请求的bgrewriteaof变为计划任务,待AOF子进程结束后执行rewrite

server.aof_last_fsync = time(NULL); //最近fsync数据到硬盘时间

server.aof_rewrite_time_last = -1;  //rewrite持续的时间

server.aof_rewrite_time_start = -1; //rewrite开始的时间

server.aof_lastbgrewrite_status = REDIS_OK; //rewrite后的状态

server.aof_delayed_fsync = 0; //延迟fsync到硬盘的次数

server.aof_fd = -1;   //AOF文件描述符

server.aof_selected_db = -1; /* Make sure the first time will not match */

server.aof_flush_postponed_start = 0; //上次推迟fsync到硬盘的时间

server.aof_rewrite_incremental_fsync = REDIS_DEFAULT_AOF_REWRITE_INCREMENTAL_FSYNC;//rewrite AOF文件时是否采用增量式fsync

Copy after login
其中aof_current_size参数未初始化,该值的初始化在Server启动加载AOF文件数据时初始化。

 

小结

本文就是简单介绍Redis中有关AOF机制实现的相关参数的意义,方便对接下来AOF数据持久化机制的理解,AOF机制的实现比较复杂,大致分为Server启动加载AOF文件数据、将新的数据追加到AOF文件中、自动rewrite AOF文件、客户端请求BGREWRITEAOF指令四个部分,下面的博客将介绍这三个部分的实现。
 

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

How to build the redis cluster mode How to build the redis cluster mode Apr 10, 2025 pm 10:15 PM

Redis cluster mode deploys Redis instances to multiple servers through sharding, improving scalability and availability. The construction steps are as follows: Create odd Redis instances with different ports; Create 3 sentinel instances, monitor Redis instances and failover; configure sentinel configuration files, add monitoring Redis instance information and failover settings; configure Redis instance configuration files, enable cluster mode and specify the cluster information file path; create nodes.conf file, containing information of each Redis instance; start the cluster, execute the create command to create a cluster and specify the number of replicas; log in to the cluster to execute the CLUSTER INFO command to verify the cluster status; make

How to implement the underlying redis How to implement the underlying redis Apr 10, 2025 pm 07:21 PM

Redis uses hash tables to store data and supports data structures such as strings, lists, hash tables, collections and ordered collections. Redis persists data through snapshots (RDB) and append write-only (AOF) mechanisms. Redis uses master-slave replication to improve data availability. Redis uses a single-threaded event loop to handle connections and commands to ensure data atomicity and consistency. Redis sets the expiration time for the key and uses the lazy delete mechanism to delete the expiration key.

How to view all keys in redis How to view all keys in redis Apr 10, 2025 pm 07:15 PM

To view all keys in Redis, there are three ways: use the KEYS command to return all keys that match the specified pattern; use the SCAN command to iterate over the keys and return a set of keys; use the INFO command to get the total number of keys.

How to use the redis command How to use the redis command Apr 10, 2025 pm 08:45 PM

Using the Redis directive requires the following steps: Open the Redis client. Enter the command (verb key value). Provides the required parameters (varies from instruction to instruction). Press Enter to execute the command. Redis returns a response indicating the result of the operation (usually OK or -ERR).

How to read redis queue How to read redis queue Apr 10, 2025 pm 10:12 PM

To read a queue from Redis, you need to get the queue name, read the elements using the LPOP command, and process the empty queue. The specific steps are as follows: Get the queue name: name it with the prefix of "queue:" such as "queue:my-queue". Use the LPOP command: Eject the element from the head of the queue and return its value, such as LPOP queue:my-queue. Processing empty queues: If the queue is empty, LPOP returns nil, and you can check whether the queue exists before reading the element.

How to start the server with redis How to start the server with redis Apr 10, 2025 pm 08:12 PM

The steps to start a Redis server include: Install Redis according to the operating system. Start the Redis service via redis-server (Linux/macOS) or redis-server.exe (Windows). Use the redis-cli ping (Linux/macOS) or redis-cli.exe ping (Windows) command to check the service status. Use a Redis client, such as redis-cli, Python, or Node.js, to access the server.

What to do if redis-server can't be found What to do if redis-server can't be found Apr 10, 2025 pm 06:54 PM

Steps to solve the problem that redis-server cannot find: Check the installation to make sure Redis is installed correctly; set the environment variables REDIS_HOST and REDIS_PORT; start the Redis server redis-server; check whether the server is running redis-cli ping.

How to read the source code of redis How to read the source code of redis Apr 10, 2025 pm 08:27 PM

The best way to understand Redis source code is to go step by step: get familiar with the basics of Redis. Select a specific module or function as the starting point. Start with the entry point of the module or function and view the code line by line. View the code through the function call chain. Be familiar with the underlying data structures used by Redis. Identify the algorithm used by Redis.

See all articles