How to use the thread pool map() method in Python to pass a multi-parameter list

The thread pool map() method passes a multi-parameter list

Previously, threading.thread() was used to facilitate multi-threaded concurrency of the interface, but this is more useful when the number of concurrencies is small. If concurrency The number is large. In addition to the processing method of thread package coroutine, we can also use the thread pool method.

In layman's terms, the implementation of the thread pool is to put all tasks in the message queue, start multiple threads and then execute the threads. However, after the thread execution is completed, the thread tasks will not be interrupted and will continue to be obtained from the message queue. Thread tasks are executed in threads, so that the thread pool saves many steps of creating and closing threads compared to multi-threaded operations, saving most resources and time.

Thread pool concurrency requires the introduction of modules

import concurrent.futures

ThreadPoolExecutor There are two thread pool methods map() and submit(). Today we will talk about the map() method

its syntax For

 with concurrent.futures.ThreadPoolExecutor() as pool:
      res = pool.map(craw, uid_list)
      print(res)

• map(), crawl is the method name, and the method name here does not contain ()

• uid_list is a method parameter, the list data type needs to be passed in the map() method

Let’s take a look at the overall code first

5000 user concurrency assistance

    def test_case_09(self):
        """5000用户并发助力"""
        # 通过yaml配置文件封装方法 获取uid_list
        uid_list = YamlHandler(YamlThePath().number_new).get_uid_list()
        # add_ticket获取5000账号登陆状态
        with concurrent.futures.ThreadPoolExecutor() as pool:
            pool.map(AccountAccess().add_ticket, uid_list)
        # 5000账号线程池方法助力用户
        with concurrent.futures.ThreadPoolExecutor() as pool:
            pool.map(PreheatMethod().help, [(uid, self.A, 1) for uid in uid_list])
        # 获取用户被助力次数
        response = PreheatMethod().init(self.A)
        print(f"当前用户被助力次数 :{response['data']['userInfo']['helpedCount']}次")

Let’s take a look at the methods of the two interfaces to get a better understanding

The first is to get the login status add_ticket

    def add_ticket(self, uid):
        """
        获取单独用户t票
        :param uid: 单独用户uid
        :return:
        """
        self.data['url'] = ApiAddress().get_ticket
        self.data['host'] = ApiAddress().host
        self.params['uid'] = str(uid)
        self.params['type'] = 0
        self.data['params'] = json.dumps(self.params)
        res = r().post(url=ApiAddress().ticket, data=self.data)
        print(f'获取t票结果：{uid}{res}')
        return uid

A very simple interface request input parameter only has one uid, but pay attention The uid here is not a list, it is just a parameter.

Then some students will have questions. The method parameter passed in map() is a list of uid content.

The map() method is to store the parameters you need in the list and request the interface you specify through traversal.

Some people may ask at this time, because I asked myself the same question at the time, what if there are multiple parameters in a method, and some of these parameters are not even fixed content.

Let’s take a look at another method of requesting the help interface

    def help(self, agrs):
        """
        助力用户
        :param agrs: uid:当前用户uid to_uid:助力用户uid count:助力次数
        :return:
        """
        uid, to_uid, count = agrs
        self.attrs['toUid'] = str(to_uid)
        self.attrs['count'] = count
        response = r().response(uid, self.code, "help", **self.attrs)
        logger.info(f'help response uid:{uid} to_uid:{to_uid}\n{response}')
        return response

Yes, we pass it to the help interface through tuples, and assign the specified keyword positions to the specified ones through the tuple. Element assignment.

In the code of the thread pool, we use list derivation to facilitate the parameters in the uid_list into the tuple you specify. Of course, if there are multiple parameters here, you can also use a dictionary to facilitate the dictionary key and value as changing parameters, because the list comprehension returns you a list, so we put the required parameters in the tuple, and the tuple in the list, so that we can use map() for multi-parameter methods. The thread pool is concurrent.

with concurrent.futures.ThreadPoolExecutor() as pool:
            pool.map(PreheatMethod().help, [(uid, self.A, 1) for uid in uid_list])

rrree

After the list derivation is obtained, it is probably the list data content format below

The above is the detailed content of How to use the thread pool map() method in Python to pass a multi-parameter list. For more information, please follow other related articles on the PHP Chinese website!