What is the process of Redis request processing?

Overview

• #The first is to register the processor;

• Open the loop listening port, and create a Goroutine every time a connection is monitored ;

• Then the Goroutine will wait in a loop to receive the request data, and then match the corresponding processor in the processor routing table according to the requested address, and then hand the request to the processor for processing ;

It’s expressed in code like this:

func (srv *Server) Serve(l net.Listener) error { 
    ...
    baseCtx := context.Background()  
    ctx := context.WithValue(baseCtx, ServerContextKey, srv)
    for {
        // 接收 listener 过来的网络连接
        rw, err := l.Accept()
        ... 
        tempDelay = 0
        c := srv.newConn(rw)
        c.setState(c.rwc, StateNew) 
        // 创建协程处理连接
        go c.serve(connCtx)
    }
}

It’s a little different for Redis, because it is single-threaded and cannot use multi-threading to handle connections. Therefore, Redis chooses to use an event driver based on the Reactor pattern to implement concurrent processing of events.

The so-called Reactor mode in Redis is to monitor multiple fds through epoll. Whenever these fds respond, epoll will be notified in the form of events for callbacks. Each Each event has a corresponding event handler.

For example: accept corresponds to the acceptTCPHandler event handler, read & write corresponds to the readQueryFromClient event handler, etc., and then the event is assigned to the event processor for processing through the event loop dispatch.

So the above Reactor mode is implemented through epoll. For epoll, there are mainly three methods:

//创建一个epoll的句柄，size用来告诉内核这个监听的数目一共有多大
int epoll_create(int size)；

/*
 * 可以理解为，增删改 fd 需要监听的事件
 * epfd 是 epoll_create() 创建的句柄。
 * op 表示 增删改
 * epoll_event 表示需要监听的事件，Redis 只用到了可读，可写，错误，挂断 四个状态
 */
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event)；

/*
 * 可以理解为查询符合条件的事件
 * epfd 是 epoll_create() 创建的句柄。
 * epoll_event 用来存放从内核得到事件的集合
 * maxevents 获取的最大事件数
 * timeout 等待超时时间
 */
int epoll_wait(int epfd, struct epoll_event * events, int maxevents, int timeout);

So we can implement a simple method based on these three methods Server:

// 创建监听
int listenfd = ::socket();

// 绑定ip和端口
int r = ::bind();  
// 创建 epoll 实例
int epollfd = epoll_create(xxx); 
// 添加epoll要监听的事件类型
int r = epoll_ctl(..., listenfd, ...);
 
struct epoll_event* alive_events =  static_cast<epoll_event*>(calloc(kMaxEvents, sizeof(epoll_event)));

while (true) {
    // 等待事件
    int num = epoll_wait(epollfd, alive_events, kMaxEvents, kEpollWaitTime);
	// 遍历事件，并进行事件处理
    for (int i = 0; i < num; ++i) {
        int fd = alive_events[i].data.fd;
        // 获取事件
        int events = alive_events[i].events;
		// 进行事件的分发
        if ( (events & EPOLLERR) || (events & EPOLLHUP) ) {
            ...
        } else  if (events & EPOLLRDHUP) {
            ...
        } 
        ...
    }   
}

Calling process

#So according to the above introduction, you can know that for Redis, an event loop is nothing more than a few steps:

• Register event listening and callback functions;

• Loop to wait for events to be acquired and processed;

• Call the callback function to process data logic;

• Write data back to Client;

• Register fd to epoll, And set the callback function acceptTcpHandler. If there is a new connection, the callback function will be called;

• Start an infinite loop to call epoll_wait to wait and continue to process the event. Later we will return to the aeMain function to loop the call aeProcessEvents function;

• When a network event comes, the callback function acceptTcpHandler will be called all the way to readQueryFromClient for data processing. readQueryFromClient will parse the client's data and find the corresponding cmd function. Execution;

• After receiving the client request, the Redis instance will process the client command and write the returned data into the client output buffer instead of returning immediately;

• Then the beforeSleep function will be called every time the aeMain function loops to write the data in the buffer back to the client;

The entire event above In fact, the code steps of the loop process have been written very clearly, and there are many articles on the Internet about it, so I won’t go into details.

Command Execution Process & Writeback Client

#Command Execution

# Let’s talk about something that is not mentioned in many articles on the Internet and see how Redis executes commands. , then store it in the cache, and write the data back to the client from the cache.

We also mentioned in the previous section that if a network event comes, the readQueryFromClient function will be called, which is where the command is actually executed. We will just follow this method and look down:

• readQueryFromClient will call the processInputBufferAndReplicate function to process the requested command;

• In the processInputBufferAndReplicate function It will call processInputBuffer and determine whether the command needs to be copied to other nodes if it is cluster mode;

• processInputBuffer function will loop through the requested command and call it according to the requested protocol processInlineBuffer function, after calling the redisObject object, processCommand is called to execute the command;

• processCommand will use lookupCommand to find the corresponding command according to the command in the server.commands table when executing the command. Execute the function, and then after a series of verifications, call the corresponding function to execute the command, and call addReply to write the returned data into the client output buffer;

server. commands will register all Redis commands in the populateCommandTable function as a table that obtains command functions based on the command name.

For example, to execute the get command, the getCommand function will be called:

void getCommand(client *c) {
    getGenericCommand(c);
}

int getGenericCommand(client *c) {
    robj *o;
	// 查找数据
    if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.nullbulk)) == NULL)
        return C_OK;
    ...
}

robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply) {
    //到db中查找数据
    robj *o = lookupKeyRead(c->db, key);
    // 写入到缓存中
    if (!o) addReply(c,reply);
    return o;
}

Find the data in the getCommand function, and then call addReply to write the returned data into the client output buffer .

Data write-back client

#After writing the command into the buffer, the data needs to be taken out from the buffer and returned to the client. For the process of writing data back to the client, it is actually completed in the event loop of the server.

• First of all, Redis will call the aeSetBeforeSleepProc function in the main function to register the function beforeSleep of the writeback package into the eventLoop;

• Then when Redis calls the aeMain function for the event loop, it will determine whether beforesleep has been set. If so, it will call it;

• beforesleep function will call Go to the handleClientsWithPendingWrites function, which calls writeToClient to write data back to the client from the buffer.

The above is the detailed content of What is the process of Redis request processing?. For more information, please follow other related articles on the PHP Chinese website!