nginx inter-process communication-socketpair

In nginx, a full-duplex communication method-socketpair is used between the master process and the worker process. After the socketpair function is successfully executed, a pair of sockets with an established connection will be created. Two processes communicating with each other can use one of the sockets to perform read and write operations respectively, thereby enabling communication between the two processes.

Looking at the nginx source code, you can see that the following function creates a socketpair

ngx_pid_t
ngx_spawn_process(ngx_cycle_t *cycle, ngx_spawn_proc_pt proc, void *data,
    char *name, ngx_int_t respawn)
{
    u_long     on;
    ngx_pid_t  pid;
    ngx_int_t  s;

    /. ......省略...... ./

    if (respawn != NGX_PROCESS_DETACHED) {

        /* Solaris 9 still has no AF_LOCAL */

        //创建socketpair
        if (socketpair(AF_UNIX, SOCK_STREAM, 0, ngx_processes[s].channel) == -1)
        {
            ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno,
                          "socketpair() failed while spawning \"%s\"", name);
            return NGX_INVALID_PID;
        }

        //非阻塞
        if (ngx_nonblocking(ngx_processes[s].channel[0]) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        //非阻塞
        if (ngx_nonblocking(ngx_processes[s].channel[1]) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        //异步
        on = 1;
        if (ioctl(ngx_processes[s].channel[0], FIOASYNC, &on) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        //设置将要在文件描述词fd上接收SIGIO 或 SIGURG事件信号的进程或进程组标识 。
        if (fcntl(ngx_processes[s].channel[0], F_SETOWN, ngx_pid) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        //设置close_on_exec，当通过exec函数族创建了新进程后，原进程的该socket会被关闭
        if (fcntl(ngx_processes[s].channel[0], F_SETFD, FD_CLOEXEC) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        //设置close_on_exec，当通过exec函数族创建了新进程后，原进程的该socket会被关闭
        if (fcntl(ngx_processes[s].channel[1], F_SETFD, FD_CLOEXEC) == -1) {
            ngx_close_channel(ngx_processes[s].channel, cycle->log);
            return NGX_INVALID_PID;
        }

        ngx_channel = ngx_processes[s].channel[1];

    } else {
        ngx_processes[s].channel[0] = -1;
        ngx_processes[s].channel[1] = -1;
    }

    ngx_process_slot = s;


    pid = fork();

    switch (pid) {

    case -1:
        ngx_close_channel(ngx_processes[s].channel, cycle->log);
        return NGX_INVALID_PID;

    case 0:
        //fork成功，子进程创建，同时相关socket描述符也会被复制一份
        ngx_pid = ngx_getpid();
        proc(cycle, data);
        break;

    default:
        break;
    }

    /. ......省略...... ./

    return pid;
}

After the fork is successful, the descriptor of the original process will also be copied. If the descriptor is no longer used during the fork process, we need to promptly closure.

If we use the fork->exec function family to create a new process, we can use better methods to ensure that the original descriptor is closed normally to avoid resource leaks. That is, the fcntl(FD_CLOEXEC) function is called on the socket in the above code and the attributes of the socket are set: when the exec function family is called, the socket will be automatically closed. Using this method of setting the FD_CLOEXEC attribute immediately after the socket is created avoids the need to manually close the relevant socket before the exec creation process, which is very practical especially when a large number of descriptors are created and managed.

Many times we use the following method to operate:

#include <sys/types.h>
#include <sys/socket.h>

#include <stdlib.h>
#include <stdio.h>

int main()
{
    pid_t  pid;
    int    fds[2];
    int    valRead, valWrite;

    if (0 > socketpair(AF_UNIX, SOCK_STREAM, 0, fds))
    {
        return 0;
    }

    pid = fork();

    if (0 == pid)
    {
        pid = getpid();
        printf("[%d]-child process start", pid);
        close(fds[0]);
        
        //read write on fds[1]
        write(fds[1], &valWrite, sizeof(valWrite)); 
        read(fds[1], &valRead, sizeof(valRead));
    }
    else if (0 < pid)
    {
        pid = getpid();
        printf("[%d]-parent process continue", pid);
        close(fds[1]);

        //read write on fds[0]
        read(fds[0], &valRead, sizeof(valRead));
        write(fds[0], &valWrite, sizeof(valWrite)); 
    }
    else
    {
        printf("%s", "fork failed");
    }

    return 0;
}

You can see that before fork, the current process created a pair of sockets, that is, socketpair. For this pair of sockets, one can be regarded as the server-side fds[0] and the other is the client-side fds[1]. Through the link established between fds[0] and fds[1], we can complete full-duplex communication. .

After fork is executed, a child process is created. In the child process, the socketpair created by the parent process will naturally be copied as fds' and exist in the child process. The parent process continues execution. At this time, the same socketpair will exist in the parent process and the child process.

Imagine that we write data to fds[0] in the main process, and the data will be read on fds'[1] in the child process, thus realizing communication between the parent process and the child process . Of course, when data is written on the main process fds[1], the written data will also be read on the sub-process fds'[0]. In our actual use, we only need to reserve a pair of sockets for communication, and the other two sockets can be closed in the parent process and the child process respectively.

Of course, if we can pass a socket in fds to another process in some way, then socketpair inter-process communication can also be achieved.

The above introduces the nginx inter-process communication-socketpair, including the relevant content. I hope it will be helpful to friends who are interested in PHP tutorials.