python3.x - python3.5怎样控制线程的数量呢？

Question

thread_list = []
for kw_do in exc_kw():

thread_list.append(Thread(target=zz_kw,args=(kw_do,)))

for thread in thread_list:

thread.start()

for thread in thread_list:

thread.join()

这样运行会卡死，请问怎样控制5个线程或者10个线程呢？

Answer 1

pool = threadpool.ThreadPool(poolSize)
poolSize is here to control the number of threads

import threadpool
import requests

def get_url(url):
    r = requests.get(url)
    return url, r.status_code

def print_result(request, result):
    print result

urls = [
    'http://www.baidu.com',
    'http://www.jd.com',
    'http://www.taobao.com',
    'https://segmentfault.com',
    'http://www.baidu.com',
    'http://www.jd.com',
    'http://www.taobao.com',
    'https://segmentfault.com',
    'http://www.baidu.com',
    'http://www.jd.com',
    'http://www.taobao.com',
    'https://segmentfault.com'
]

pool = threadpool.ThreadPool(5)

for th in threadpool.makeRequests(get_url, urls, print_result):
    pool.putRequest(th)

pool.wait()

Answer 2

Your problem is probably that there will be too many threads when there are many tasks. To look at this problem differently, use one thread, set it to 5 or 10, and then throw the task into the thread pool. Refer to the usage of python ThreadPoolExecutor.

Answer 3

It is recommended to use multiple processes to try to see if the stuck phenomenon will also occur!
Try to find out the cause of the stuck, instead of immediately controlling the number of threads to hide the stuck problem!