linux - IO阻塞非阻塞、同步异步有什么区别？

Question

IO流程是怎样的？nginx、apache是基于哪种IO，为什么？

Answer 1

You can use sudo strace -p PID to view the system calls of the process with the numbered PID.
nginx master: rt_sigsuspend
nginx worker: epoll_wait
php-fpm master: epoll_wait (events.mechanism = epoll)
php-fpm worker: accept poll
It can be seen that the Nginx worker process made the epoll_wait system call. epoll is an asynchronous network IO programming interface provided by the Linux kernel.

http://liuxun.org/blog/nginx-gong-zuo-jin-cheng-mo-xing/
Nginx does not use the master process to distribute connections like PHP-FPM. This work is done by the operating system The kernel mechanism is completed,
so it may cause a thundering herd phenomenon, that is, when listen_fd has a new accept() request, the operating system will wake up all child processes.
Nginx uses a global mutex lock to avoid a thundering herd ( accept_mutex on), each worker process applies for a lock before epoll_wait(). If the application is received, it will continue processing. If it cannot be obtained, it will wait.
And a load balancing algorithm is set up (when the task volume of a certain worker process When it reaches 7/8 of the total settings, it will no longer try to apply for locks) to balance the task load of each process.
http://nginx.org/en/docs/ngx_core_module.html#accept_mutex

Nginx’s new method to solve thundering herds: use the Socket ReusePort function provided by the kernel
NGINX 1.9.1 supports socket sharding:
http://nglua.com/docs/sharding.html
http ://nginx.com/blog/socket-sharding-nginx-release-1-9-1/
NGINX1.9.1 supports the SO_REUSEPORT option of socket. This option is valid in new versions of many operating systems, including DragonFly BSD and Linux (3.9+ kernel).
This option allows multiple sockets to listen to the same IP address and port combination. The kernel load balances these incoming socket connections and effectively fragments these sockets.
When the SO_REUSEPORT option is not available When enabled, the listening socket will notify a certain process by default when a connection comes in.
If the accept_mutex off command will wake up all the working processes, they will compete to get it. This is the so-called thundering herd phenomenon.
If you use epoll without locking (accept_mutex off), when there is a read operation on the listening port, a thundering herd phenomenon will occur.
After enabling the SO_REUSEPORT option, each process will have an independent listening socket. The kernel determines Which is the valid socket (process) to get the connection?
Doing this reduces latency and improves the performance of the worker process, it also means that the worker process is given new connections before it is ready to handle them.
Enable SO_REUSEPORT is supported, just add the new parameter reuseport to the end of the listen command:
listen 80 reuseport;
Including this reuseport parameter will disable the accept_mutex of the listening socket, because the lock becomes redundant.
Benchmark test: After enabling reuseport in NGINX 1.9.1 to perfectly solve the problem of thundering groups, the number of requests processed per second increased by 2 to 3 times, while reducing the delay and stdev indicators.
Attachment: Taobao's Tengine is said to have joined very early Socket fragmentation function.

The above talks about the asynchronousness of network IO, and the following talks about the asynchronousness of disk IO.

In high-performance server programming, the I/O model is of course a top priority and needs to be carefully selected.
For network sockets, we can use epoll to poll, although epoll also has There are some flaws, but overall it is very efficient, especially in scenarios with a large number of sockets.
But it cannot be used for Regular File. Use poll/epoll, that is, O_NOBLOCK method for traditional file handles is invalid.
That is to say, our Regular File operations such as open, read, and mkdir will definitely cause blocking.
In the multi-thread and multi-process model, you can choose to perform IO in a synchronous blocking manner. Operation and task scheduling are ensured by the operating system to ensure fairness.

NGINX experimentally introduces thread pools from 1.7.11, and the performance in specific scenarios is improved by 9 times.
https://www.nginx.com/blog/thread-pools-boost-performance-9x/
Test The server has 2 Intel Xeon E5645 processors (total: 12 cores, 24 hyper-threads) and 10-Gbps network interface.
The disk subsystem is a RAID10 array composed of 4 Western Digital WD1003FBYX disks.
Operation The system is Ubuntu Server 14.04.1 LTS.

thread_pool default threads=32 max_queue=65536;
aio threads=default;
Here defines a thread pool named default, containing 32 threads, and the task queue supports up to 65536 requests.
If the task queue is overloaded, NGINX will output the following error log and reject the request:
thread pool "NAME" queue overflow: N tasks waiting

With this thread pool, it is possible for NGINX to offload any long-blocking operations without any performance penalty.
Many popular libraries still did not provide asynchronous non-blocking interfaces, previously, which made them incompatible with NGINX.
We could spend a lot of time and resources developing our own non-blocking prototype libraries, but is it always worth it?
Now, with thread pools, we can use these libraries relatively easily, without affecting the performance of these modules.
FreeBSD already has a good enough asynchronous interface to read files, and there is no need to use a thread pool at this time.
Linux lacks such a mechanism, so for some files that are not suitable for caching in the virtual For files in memory (large files), read operations can be offloaded to the AIO thread pool to avoid blocking the worker process.
It would be very meaningful if we could improve offloading read operations to the thread pool.
We only need to know all the Check whether the required file data is in memory. Only when it is not in memory, the read operation should be offloaded to a separate thread.

The asynchronous file operation interface AIO provided by the Linux kernel requires DirectIO and cannot utilize the memory Page Cache.
This strange implementation may be specially designed by Oracle/IBM for database applications, and MySQL uses AIO.
AIO+DirectIO bypasses the virtual file system VFS cache. Of course, large database systems such as Oracle and MySQL (InnoDB) are very happy.
Because Oracle and MySQL have their own cache systems, there is no need for the operating system's Page Cache cache, DirectIO can avoid data being cached twice and wasting memory.
In other words, if you want your program to implement asynchronous file IO through AIO, then you'd better build your own memory cache system instead of Depends on the kernel.

Finally, the default event MPM of the Apache 2.4 series on Linux is a multi-process epoll event-driven MPM with each worker process containing multiple threads.
The prefork MPM of the Apache 2.2 series is a pure multi-process architecture , does not introduce multi-threading, and does not use the epoll feature of the kernel. It is probably mainly for the convenience of transplantation.
The IBM HTTP Server on IBM AIX seems to be modified based on Apache.