Master the importance and practical skills of Linux processes and threads

As a Linux system administrator, it is very important to understand and master the concepts of processes and threads and the corresponding practical skills. This not only helps us better manage system resources, but also improves the operating efficiency and stability of the system. This article will give you an in-depth understanding of Linux processes and threads, and share some practical tips and tools.

A process is a running activity of a program with certain independent functions on a certain data collection. It is the basic unit of dynamic execution of the operating system. In traditional operating systems, the process is both the basic allocation unit and the basic execution unit.

process

A process is a running activity of a program with certain independent functions on a certain data collection. It is the basic unit of dynamic execution of the operating system. In traditional operating systems, the process is both the basic allocation unit and the basic execution unit. There are two main points in the concept of process: First, a process is an entity. Each process has its own address space, which generally includes text region, data region and stack region. The text area stores code executed by the processor; the data area stores variables and dynamically allocated memory used during process execution; and the stack area stores instructions and local variables for active procedure calls. Second, a process is an "executing program." A program is an inanimate entity. Only when the processor gives life to the program (executed by the operating system) can it become an active entity. We call it a process.

Thread

Thread is the smallest unit that the operating system can perform operation scheduling. It is included in the process and is the actual operating unit in the process. A thread refers to a single sequential control flow in a process. Multiple threads can run concurrently in a process, and each thread performs different tasks in parallel. It is also called lightweight processes in Unix System V and SunOS, but lightweight processes refer more to kernel threads, while user threads are called threads. The relationship between processes and threads Multiple threads in the same process will share all system resources in the process, such as virtual address space, file descriptors, signal processing, etc. But multiple threads in the same process have their own call stacks, their own register context, and their own thread-local storage.

Threads and processes in Linux

In the Linux kernel, although processes and threads are both tasks, they should be distinguished. Where pid is the process id and tgid is the thread group ID. For any process, if there is only the main thread, then the pid is itself, the tgid is itself, and the group_leader still points to itself. However, if a process creates other threads, this changes. The thread has its own pid, tgid is the pid of the main thread of the process, and group_leader points to the main thread of the process. So with tgid, we know whether taste_struct represents a process or a thread. The relationship is as follows:

About kernel parameters of threads and processes

ulimit limit, execute ulimit -a under Linux, you will see ulimit limits on various resources.

The "max user processes" is the maximum number of threads that a process can create. We can modify this parameter:

ulimit -u 66535

2. Parameter sys.kernel.threads-max limit. This parameter limits the number of threads globally in the operating system. You can view its value through the following command. How to view threads-max:

cat /proc/sys/kernel/threads-max
32768

Method to modify this value:

#方法一，重启后会失效
echo 65535 > /proc/sys/kernel/threads-max
#方法二，永久修改
echo "kernel.threads-max = 65535" >> /etc/sysctl.conf

3. Parameter sys.kernel.pid_max limit. This parameter limits the number of threads globally in the operating system. You can view its value through the following command. Let me talk about the 32-bit operating system. The maximum value is 32768 and cannot be modified. The maximum value of pid_max on the 64-bit system is 2^22. When the Linux kernel initializes the system, it will set the pid_max value according to the number of CPUs in the machine. For example, if the number of CPUs in the machine is less than or equal to 32, then pid_max will be set to 32768 (32K); if the number of CPUs in the machine is greater than 32, then pid_max will be set to N*1024 (N is the number of CPUs). How to check pid_max:

cat /proc/sys/kernel/pid_max
32768

Method to modify this value:

#方法一，重启后会失效
echo 65535 > /proc/sys/kernel/pid_max
#方法二，永久修改
echo "kernel.pid_max = 65535" >> /etc/sysctl.conf

Note: A number of threads will also occupy a pid, so threads-max needs to be less than or equal to pid_max.

Limit on the number of container threads

For Linux systems, a container is a collection of processes. If the application in the container creates too many processes or has bugs, it will produce behavior similar to fork bombs. In this way, not only will other containers on the same node be unable to work, but the host itself will also be unable to work. So for each container, we need to limit its maximum number of processes, and this function is completed by the pids Cgroup subsystem. I have encountered such a problem before. Because Java applications have to handle many scheduled tasks, a scheduled task pulls up a thread. However, due to bugs in the code, the threads were not recycled in time, and then the container continued to generate threads, which exhausted the host's process table space, eventually causing the entire Linux service to report the error "java.lang.OutOfMemoryError: Unable to create native threads", affecting other services. An error message of "Resource temporarily unavailable" appears during the creation process. In addition to allowing developers to fix bugs, this kind of problem also requires limiting the number of threads at the system level.

cgroup

Pids are isolated in cgroup. By changing the docker/kubelet configuration, you can limit the total number of pids to limit the total number of threads.

docker, set the –pids-limit parameter when starting the container to limit the total number of container-level pids

kubelet, enable the SupportPodPidsLimit feature, set the –pod-max-pids parameter, and limit the total number of pids for each pod of the node

The principle is as follows: After a container is created, the service that creates the container will create a subdirectory under /sys/fs/cgroup/pids, which is a control group. The most critical file in the control group is pids.max. Kubelet or docker writes a value to this file, and this value is the maximum number of processes allowed in this container. Each node in Kubernetes runs a service called Kubelet, which is responsible for the status and life cycle of the containers on the node, such as creating and deleting containers. According to the official documentation of Kubernetes, Process ID Limits And Reservations, you can set the –pod-max-pids configuration option of the Kubelet service, and then the containers created on the node will eventually use the Cgroups pid controller to limit the number of processes in the container.

Summarize

Through the introduction and analysis of this article, we understand the concepts, differences, state transitions and their impact on system resources of Linux processes and threads. At the same time, we also shared some practical tips and tools, such as process tuning, detection tools, and management tools. These tools and techniques can not only help us better manage system resources and improve the operating efficiency and stability of the system, but also improve our work efficiency and competitiveness. Therefore, it is very necessary for us to master the importance and practical skills of Linux processes and threads.

The above is the detailed content of Master the importance and practical skills of Linux processes and threads. For more information, please follow other related articles on the PHP Chinese website!