This article brings you a tutorial on python multi-process control (with examples). It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
Introduction to multiprocessing
Multiprocessing is a multi-process module that comes with python. It can generate processes in large batches. The effect is better when the server is a multi-core CPU, similar to threading. module. Compared with multi-threading, multi-process is more stable and secure due to its exclusive memory space. When doing batch operations in operation and maintenance, multi-process has more applicable scenarios
The multiprocessing package provides both local and remote Concurrent operations effectively avoid using child processes instead of threads with global interpretation locks. Therefore, multiprocessing can effectively utilize multi-core processing
Process class
In multiporcessing, processes are generated in batches through Process class objects, and the start() method is used to start the process
1. Syntax
1 2 3 4 5 6 7 |
|
2.Methods and objects of the Process class
run()
This method is the running process of the process and can be repeated in subclasses When writing this method, it is generally rarely reconstructed
start()
Start the process, each process object must be called by this method
join([timeout])
Wait for the process to terminate before continuing. You can set the timeout period
name
You can get the process name, multiple processes can also have the same name
is_alive()
Returns whether the process is still alive, True or False, process survival refers to the start of start() to the termination of the child process
daemon
The mark of the daemon process, a Boolean value, after start() Set this value to indicate whether to run in the background
Note: If background running is set, the background program will not run and then create a child process
pid
You can get the process ID
exitcode
The value when the child process exits. If the process has not terminated, the value will be None. If it is a negative value, it means The child process is terminated
terminate()
Terminate the process. If it is Windows, use terminateprocess(). This method treats the process that has exited and ended. , will not be executed
The following is a simple example:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
Of course, the ID of each process can also be displayed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
But in the actual use process, it is not enough to complete the concurrency. For example, there are 30 tasks. Due to limited server resources, 5 tasks are concurrent each time. , this also involves the issue of how to obtain 30 tasks. In addition, it is difficult to ensure consistent execution time of concurrent process tasks, especially tasks that require time. There may be 5 tasks concurrently, 3 of which have been completed, and 2 of which still require a lot of time. For long-term execution, you cannot wait until these two processes have finished executing before continuing to execute subsequent tasks. Therefore, process control has usage scenarios here. You can use the Process method in combination with some multiprocessing packages and classes.
Common classes for process control and communication
1. Queue class
is similar to Queue.Queue that comes with python, mainly used in
Syntax for smaller queues:
1 |
|
Class method:
qsize()
Returns the approximate size of the queue, because multiple processes or threads have been consuming Queue, so the data is not necessarily correct
empty()
Determine whether the queue is empty, if so, return True, otherwise False
full()
Determine whether the queue is full, if so, return True, otherwise False
put(obj[, block[, timeout]])
Put the object into the queue, the optional parameter block is True, and the timeout is None
get()
Remove the object from the queue
1 2 3 4 5 6 7 8 9 10 11 12 |
|
二, Pipe class
pipe() function returns a pair of object connections, which can transmit messages between processes. It is useful for printing some logs and process control. The Pip() object returns two objects. Connection represents two channels. Each connection object has send() and recv() methods. It should be noted that two or more processes read or write to the same pipe at the same time, which may cause data confusion. I tested it. , is directly covered. In addition, if one of the two connections returned is send() data, then the other one can only receive data with recv()
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
|
The output is:
三、Value,Array
在进行并发编程时,应尽量避免使用共享状态,因为多进程同时修改数据会导致数据破坏。但如果确实需要在多进程间共享数据,multiprocessing也提供了方法Value、Array
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
*print
3.1415927
[0, -1, -2, -3, -4, -5, -6, -7, -8, -9]*
四、Manager进程管理模块
Manager类管理进程使用得较多,它返回对象可以操控子进程,并且支持很多类型的操作,如: list, dict, Namespace、lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value, Array,因此使用Manager基本上就够了
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
输出:
{0.25: None, 1: '1', '2': 2}
[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
可以看到,跟共享数据一样的效果,大部分管理进程的方法都集成到了Manager()模块了
五、对多进程控制的应用实例
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
|
The above is the detailed content of Tutorial explanation of python multi-process control (with examples). For more information, please follow other related articles on the PHP Chinese website!