When to use multi-process programming in python

In concurrent programming, multi-threading and multi-process are two modes that are often used (in addition to coroutines, etc.). Due to CPython's GIL limitations (Jython and IronPython do not have GIL, PyPy is trying to remove GIL), only threads that have obtained GIL can use the CPU, so in addition to needing to handle some IO that may be blocked (reading and writing files, accessing the network, etc. ), basically no one will use CPython's multi-threading. Therefore, this article will talk about more useful Python multi-process programming.

Note: Recommended learning: Python video tutorial)

The environment described in this article is CPython under the Linux operating system (also applicable to most POSIX systems) may not be applicable to Windows operating systems or other Python implementations.

In order to avoid ambiguity, "main process" or "current process" is used below to refer to the process that created the child process, and "parent process" is not used, unless "xx's parent process" is explicitly specified.

Different from the need to pass a callable object during multi-thread programming, during multi-process programming, the main process is copied to the child process, and the child process cannot be directly required to execute a callable object.

In the POSIX system, this copy operation is completed by the clone() and fork() system calls, and the latter is generally used.
If fork() is executed successfully, the PID and 0 of the child process will be returned in the main process and the child process respectively, and then the execution code will start to be different. If it fails (not enough memory, PID reaches the upper limit, etc.), the child process will not be created, the main process will return -1, and errno will be set to the corresponding error code.
In the implementation of CPython, os.fork() is mainly an encapsulation of the fork() function. The difference is that an OSError is thrown when it fails. The errno attribute of the exception is the corresponding error code.

Therefore, the multi-process Python code will roughly look like this:

import os

try:
    pid = os.fork()
    if pid == 0:  # 子进程
        # 子进程的代码
    else:  # 主进程
        # 主进程的代码
except OSError:
    # 主进程处理 fork 失败的代码

For more Python related technical articles, please visit the Python Tutorial column Get studying!

The above is the detailed content of When to use multi-process programming in python. For more information, please follow other related articles on the PHP Chinese website!