This article brings you an introduction (code) about the concurrent future module in Python. It has certain reference value. Friends in need can refer to it. I hope it will be helpful to you.
The main feature of this module is the ThreadPoolExecutor and ProcessPoolExecutor classes. Both classes inherit from the concurrent.futures._base.Executor class. The interfaces they implement can be used in different Callable objects are executed in threads or processes, and they all maintain a worker thread or process pool internally.
The ThreadPoolExecutor and ProcessPoolExecutor classes are advanced classes. In most cases, you only need to learn to use them without paying attention to their implementation details.
#ProcessPoolExecutor class>class ThreadPoolExecutor(concurrent.futures._base.Executor) >| This is an abstract base class for concrete asynchronous executors. >| Method resolution order: >| ThreadPoolExecutor | concurrent.futures._base.Executor | builtins.object | | Methods defined here: | | init(self, max_workers=None, thread_name_prefix='') | Initializes a new ThreadPoolExecutor instance. | | Args: | max_workers: The maximum number of threads that can be used to | execute the given calls. | thread_name_prefix: An optional name prefix to give our threads. | | shutdown(self, wait=True) | Clean-up the resources associated with the Executor. | | It is safe to call this method several times. Otherwise, no other | methods can be called after this one. | | Args: | wait: If True then shutdown will not return until all running | futures have finished executing and the resources used by the | executor have been reclaimed. | | submit(self, fn, *args, **kwargs) | Submits a callable to be executed with the given arguments. | | Schedules the callable to be executed as fn(*args, **kwargs) and returns | a Future instance representing the execution of the callable. | | Returns: | A Future representing the given call. | | ---------------------------------------------------------------------- | Methods inherited from concurrent.futures._base.Executor: | | enter(self) | | exit(self, exc_type, exc_val, exc_tb) | | map(self, fn, *iterables, timeout=None, chunksize=1) | Returns an iterator equivalent to map(fn, iter). | | Args: | fn: A callable that will take as many arguments as there are | passed iterables. | timeout: The maximum number of seconds to wait. If None, then there | is no limit on the wait time. | chunksize: The size of the chunks the iterable will be broken into | before being passed to a child process. This argument is only | used by ProcessPoolExecutor; it is ignored by | ThreadPoolExecutor. | | Returns: | An iterator equivalent to: map(func, *iterables) but the calls may | be evaluated out-of-order. | | Raises: | TimeoutError: If the entire result iterator could not be generated | before the given timeout. | Exception: If fn(*args) raises for any values.
from concurrent import futures with futures.ProcessPoolExecutor(max_works=3) as executor: executor.map()
class ThreadPoolExecutor(concurrent.futures._base.Executor) | This is an abstract base class for concrete asynchronous executors. | | Method resolution order: | ThreadPoolExecutor | concurrent.futures._base.Executor | builtins.object | | Methods defined here: | | init(self, max_workers=None, thread_name_prefix='') | Initializes a new ThreadPoolExecutor instance. | | Args: | max_workers: The maximum number of threads that can be used to | execute the given calls. | thread_name_prefix: An optional name prefix to give our threads. | | shutdown(self, wait=True) | Clean-up the resources associated with the Executor. | | It is safe to call this method several times. Otherwise, no other | methods can be called after this one. | | Args: | wait: If True then shutdown will not return until all running | futures have finished executing and the resources used by the | executor have been reclaimed. | | submit(self, fn, *args, **kwargs) | Submits a callable to be executed with the given arguments. | | Schedules the callable to be executed as fn(*args, **kwargs) and returns | a Future instance representing the execution of the callable. | | Returns: | A Future representing the given call. | | ---------------------------------------------------------------------- | Methods inherited from concurrent.futures._base.Executor: | | enter(self) | | exit(self, exc_type, exc_val, exc_tb) | | map(self, fn, *iterables, timeout=None, chunksize=1) | Returns an iterator equivalent to map(fn, iter). | | Args: | fn: A callable that will take as many arguments as there are | passed iterables. | timeout: The maximum number of seconds to wait. If None, then there | is no limit on the wait time. | chunksize: The size of the chunks the iterable will be broken into | before being passed to a child process. This argument is only | used by ProcessPoolExecutor; it is ignored by | ThreadPoolExecutor. | | Returns: | An iterator equivalent to: map(func, *iterables) but the calls may | be evaluated out-of-order. | | Raises: | TimeoutError: If the entire result iterator could not be generated | before the given timeout. | Exception: If fn(*args) raises for any values.
from time import sleep, strftime from concurrent import futures def display(*args): print(strftime('[%H:%M:%S]'), end="") print(*args) def loiter(n): msg = '{}loiter({}): doing nothing for {}s' display(msg.format('\t'*n, n, n)) sleep(n) msg = '{}loiter({}): done.' display(msg.format('\t'*n, n)) return n*10 def main(): display('Script starting') executor = futures.ThreadPoolExecutor(max_workers=3) results = executor.map(loiter, range(5)) display('results:', results) display('Waiting for inpidual results:') for i, result in enumerate(results): display('result {} : {}'.format(i, result)) if __name__ == '__main__': main()
[20:32:12]Script starting [20:32:12]loiter(0): doing nothing for 0s [20:32:12]loiter(0): done. [20:32:12] loiter(1): doing nothing for 1s [20:32:12] loiter(2): doing nothing for 2s [20:32:12]results: <generator object Executor.map.<locals>.result_iterator at 0x00000246DB21BC50> [20:32:12]Waiting for inpidual results: [20:32:12] loiter(3): doing nothing for 3s [20:32:12]result 0 : 0 [20:32:13] loiter(1): done. [20:32:13] loiter(4): doing nothing for 4s [20:32:13]result 1 : 10 [20:32:14] loiter(2): done. [20:32:14]result 2 : 20 [20:32:15] loiter(3): done. [20:32:15]result 3 : 30 [20:32:17] loiter(4): done. [20:32:17]result 4 : 40
Detailed examples of how Python handles concurrency issues through futures
The above is the detailed content of Introduction to concurrent future module in Python (code). For more information, please follow other related articles on the PHP Chinese website!