This article lists four classic Linux packet collection engines. If there are others you think are OK, you can leave a message. These four are:
libpcap’s packet capture mechanism is to add a Bypass processing does not interfere with the processing of the system's own network protocol stack. The sent and received data packets are filtered and buffered through the Linux kernel, and finally passed directly to the upper-layer application.
libpcap-mmap is an improvement on the old libpcap implementation. New versions of libpcap basically use the packet_mmap mechanism. PACKET_MMAP uses mmap to reduce one memory copy ( "The fourth copy is gone" ), which reduces frequent system calls and greatly improves message capture. s efficiency.
We saw that libpcap had 4 memory copies before. libpcap_mmap has 3 memory copies. The core solution proposed by PF_RING is to reduce the number of copies of messages during transmission.
We can see that compared to libpcap_mmap, pfring allows user space memory to mmap directly with rx_buffer. This reduces another copy ( 「Second copy of libpcap_mmap」: rx_buffer->skb)
PF-RING ZC implements DNA (Direct NIC Access (direct network card access) technology maps the user memory space to the driver's memory space, allowing user applications to directly access the registers and data of the network card.
In this way, the cache of data packets in the kernel is avoided and one copy is reduced ( 「The first copy of libpcap」 , DMA to kernel buffer copy). This is completely zero copy.
The disadvantage is that only one application can open the DMA ring at a time (note that today's network cards can have multiple RX/TX queues, allowing one application to be on each queue at the same time) , In other words, multiple applications in user mode need to communicate with each other to distribute data packets.
pf-ring Both zc and dpdk can achieve zero copy of data packets. Both bypass the kernel, but the implementation principles are slightly different. PF-ring zc takes over the data packets through the zc driver (also at the application layer), and dpdk is implemented based on UIO.
UIO (Userspace I/O) is an I/O technology that runs in user space. Generally, driver devices in Linux systems run in the kernel space and can be called by applications in the user space. However, UIO runs a small part of the driver in the kernel space and implements the vast majority of the driver in the user space. Function. Using the UIO mechanism provided by Linux, the Kernel can be bypassed and all packet processing work is completed in the user space.
DPDK’s UIO driver blocks hardware-issued interrupts and then uses active polling in user mode. This The mode is called PMD (Poll Mode Driver).
Compared with DPDK, pf-ring (no zc) uses NAPI polling and application layer polling, while pf-ring zc is similar to DPDK and only uses application layer polling.
After the MMU (Memory Management Unit) is introduced into the operating system, the CPU needs to access the memory twice to read the memory data. The first time is to query the page table to convert the logical address into a physical address, and then access the physical address to read data or instructions.
In order to reduce the problem of too long query time caused by too many pages and too large page tables, TLB (Translation Lookaside Buffer) was introduced, which can be translated as an address translation buffer. The TLB is a memory management unit, generally stored in a register, which stores a small portion of the page table entries that are most likely to be accessed currently.
After the TLB is introduced, the CPU will first go to the TLB to address. Since the TLB is stored in the register and it only contains a small part of the page table entries, the query speed is very fast. If the addressing in the TLB is successful (TLB hit), there is no need to query the page table in RAM; if the addressing in the TLB fails (TLB miss), you need to query the page table in RAM. After querying, the page will be updated. into the TLB.
DPDK uses HugePages, which supports page sizes of 2MB and 1GB under x86-64, which greatly reduces the total number of pages and the size of the page table, thus greatly reducing the probability of TLB miss and improving CPU addressing performance. .
xdp represents the eXpress data path , use ebpf for packet filtering. Compared with dpdk, which sends data packets directly to user mode and uses user mode as a fast data processing plane, xdp creates a data fast plane in the driver layer. The data packet is processed before the data is dma'd by the network card hardware into the memory and skb is allocated.
Please note that XDP does not perform Kernel bypass on data packets, it just does a little pre-checking in advance.
Compared with DPDK, XDP has the following advantages:
XDP usage scenarios include:
OK, the above is today’s sharing. If you think there are other packet collection engines, you can leave a message to share.
The above is the detailed content of Several classic Linux packet collection engines. For more information, please follow other related articles on the PHP Chinese website!