multithreaded programming

In fact, after a thread is created, the thread does not always maintain a state. Its state is roughly as follows:

New creates

Runnable ready. Waiting for schedule

Running Running

Blocked blocked. Blocking may occur in Wait Locked Sleeping

Dead Death

Threads have different states and different types. It can be roughly divided into:

Main thread

Sub-thread

Daemon thread (background thread)

Foreground thread

Complete simple understanding After these, we start to look at the specific code usage.

1. Creation of threads

Python provides two modules for multi-threaded operations, namely thread and threading

The former is Relatively low-level modules are used for lower-level operations and are not commonly used in general application-level development.

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import time
import threading
class MyThread(threading.Thread):
    def run(self):
        for i in range(5):
            print('thread {}, @number: {}'.format(self.name, i))
            time.sleep(1)
def main():
    print("Start main threading")
    # 创建三个线程
    threads = [MyThread() for i in range(3)]
    # 启动三个线程
    for t in threads:
        t.start()
    print("End Main threading")
if __name__ == '__main__':
    main()

Running results:

Start main threading
thread Thread-1, @number: 0
thread Thread-2, @number: 0
thread Thread-3, @number: 0
End Main threading
thread Thread-2, @number: 1
thread Thread-1, @number: 1
thread Thread-3, @number: 1
thread Thread-1, @number: 2
thread Thread-3, @number: 2
thread Thread-2, @number: 2
thread Thread-2, @number: 3
thread Thread-3, @number: 3
thread Thread-1, @number: 3
thread Thread-3, @number: 4
thread Thread-2, @number: 4
thread Thread-1, @number: 4

Note that the output results of different environments here are definitely different.

2. Thread merging (join method)

From the results printed in the above example, after the main thread ends, the child thread is still running. So we need the main thread to wait for the child thread to finish running before exiting. What should we do?

At this time, you need to use the join method.

In the above example, add a piece of code, as follows:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import time
import threading
class MyThread(threading.Thread):
    def run(self):
        for i in range(5):
            print('thread {}, @number: {}'.format(self.name, i))
            time.sleep(1)
def main():
    print("Start main threading")
    # 创建三个线程
    threads = [MyThread() for i in range(3)]
    # 启动三个线程
    for t in threads:
        t.start()
    # 一次让新创建的线程执行 join
    for t in threads:
        t.join()
    print("End Main threading")
if __name__ == '__main__':
    main()

From the printed results, you can clearly see that compared to the results printed in the above example, the main thread is waiting It ends after the child thread has finished running.

Start main threading
thread Thread-1, @number: 0
thread Thread-2, @number: 0
thread Thread-3, @number: 0
thread Thread-1, @number: 1
thread Thread-3, @number: 1
thread Thread-2, @number: 1
thread Thread-2, @number: 2
thread Thread-1, @number: 2
thread Thread-3, @number: 2
thread Thread-2, @number: 3
thread Thread-1, @number: 3
thread Thread-3, @number: 3
thread Thread-3, @number: 4
thread Thread-2, @number: 4
thread Thread-1, @number: 4
End Main threading

3. Thread synchronization and mutex lock

Using thread loading to obtain data usually results in data being out of sync. Of course, we can lock the resource at this time, that is, the thread accessing the resource needs to obtain the lock before accessing it.

The threading module provides us with a Lock function.

lock = threading.Lock()

Acquire locks in threads

lock.acquire()

After use is completed, we definitely need to release the lock

lock.release()

Of course, in order to support multiple requests for the same resource in the same thread, Python provides A reentrant lock (RLock) is available. RLock internally maintains a Lock and a counter variable. The counter records the number of acquires, so that the resource can be required multiple times. Until all acquires of a thread are released, other threads can obtain the resource.

So how to create a reentrant lock? It’s also a matter of code:

r_lock = threading.RLock()

4. Condition condition variable

Practical locks can achieve thread synchronization, but in more complex environments, some conditions need to be implemented for the locks judge. Python provides Condition objects. The Condition object can be used to process data after certain events are triggered or specific conditions are met. In addition to the acquire method and release method of the Lock object, Condition also provides wait and notify methods. The thread first acquires a condition variable lock. If the conditions are insufficient, the thread waits. If the conditions are met, the thread is executed, and it can even notify other threads. Other threads in the wait state will re-judge the conditions after receiving the notification.

The condition variable can be seen as different threads acquiring the lock one after another. If the condition is not met, it can be understood as being thrown into a (Lock or RLock) waiting pool. Directly notify other threads and then re-judge the condition. This process is repeated continuously to solve complex synchronization problems.

This model is often used in the producer-consumer model. Specifically, look at the following examples of online shopping buyers and sellers:

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import threading, time
class Consumer(threading.Thread):
    def __init__(self, cond, name):
        # 初始化
        super(Consumer, self).__init__()
        self.cond = cond
        self.name = name
    def run(self):
        # 确保先运行Seeker中的方法
        time.sleep(1)
        self.cond.acquire()
        print(self.name + ': 我这两件商品一起买，可以便宜点吗')
        self.cond.notify()
        self.cond.wait()
        print(self.name + ': 我已经提交订单了，你修改下价格')
        self.cond.notify()
        self.cond.wait()
        print(self.name + ': 收到，我支付成功了')
        self.cond.notify()
        self.cond.release()
        print(self.name + ': 等待收货')
class Producer(threading.Thread):
    def __init__(self, cond, name):
        super(Producer, self).__init__()
        self.cond = cond
        self.name = name
    def run(self):
        self.cond.acquire()
        # 释放对琐的占用，同时线程挂起在这里，直到被 notify 并重新占有琐。
        self.cond.wait()
        print(self.name + ': 可以的，你提交订单吧')
        self.cond.notify()
        self.cond.wait()
        print(self.name + ': 好了，已经修改了')
        self.cond.notify()
        self.cond.wait()
        print(self.name + ': 嗯，收款成功，马上给你发货')
        self.cond.release()
        print(self.name + ': 发货商品')
cond = threading.Condition()
consumer = Consumer(cond, '买家（两点水）')
producer = Producer(cond, '卖家（三点水）')
consumer.start()
producer.start()

The output results are as follows:

买家（两点水）: 我这两件商品一起买，可以便宜点吗
卖家（三点水）: 可以的，你提交订单吧
买家（两点水）: 我已经提交订单了，你修改下价格
卖家（三点水）: 好了，已经修改了
买家（两点水）: 收到，我支付成功了
买家（两点水）: 等待收货
卖家（三点水）: 嗯，收款成功，马上给你发货
卖家（三点水）: 发货商品

5. Inter-thread communication

If there are multiple threads in the program, these threads will inevitably need to communicate with each other. So how do we exchange information or data safely between these threads?

Perhaps the safest way to send data from one thread to another is to use a queue in the queue library. Creates a Queue object that is shared by multiple threads, which add or remove elements from the queue using put() and get() operations.

# -*- coding: UTF-8 -*-
from queue import Queue
from threading import Thread
isRead = True
def write(q):
    # 写数据进程
    for value in ['两点水', '三点水', '四点水']:
        print('写进 Queue 的值为：{0}'.format(value))
        q.put(value)
def read(q):
    # 读取数据进程
    while isRead:
        value = q.get(True)
        print('从 Queue 读取的值为：{0}'.format(value))
if __name__ == '__main__':
    q = Queue()
    t1 = Thread(target=write, args=(q,))
    t2 = Thread(target=read, args=(q,))
    t1.start()
    t2.start()

The output results are as follows:

写进 Queue 的值为：两点水
写进 Queue 的值为：三点水
从 Queue 读取的值为：两点水
写进 Queue 的值为：四点水
从 Queue 读取的值为：三点水
从 Queue 读取的值为：四点水

Python also provides the Event object for inter-thread communication. It is a signal flag set by the thread. If the signal flag is true, other threads wait until signal contact.

The Event object implements a simple thread communication mechanism. It provides setting signals, clearing signals, waiting, etc. for realizing communication between threads.

Set the signal

Use the set() method of Event to set the signal flag inside the Event object to true. The Event object provides the isSe() method to determine the status of its internal signal flag. When the set() method of the event object is used, the isSet() method returns true

Clear the signal

Use the clear() method of the Event object to clear the signal flag inside the Event object , that is, set it to false. When the clear method of Event is used, the isSet() method returns false.

Waiting

The wait method of the Event object only works when the internal signal is true. Only then will it be executed and returned quickly. When the internal signal flag of the Event object is false, the wait method waits until it is true before returning.

Example:

# -*- coding: UTF-8 -*-
import threading
class mThread(threading.Thread):
    def __init__(self, threadname):
        threading.Thread.__init__(self, name=threadname)
    def run(self):
        # 使用全局Event对象
        global event
        # 判断Event对象内部信号标志
        if event.isSet():
            event.clear()
            event.wait()
            print(self.getName())
        else:
            print(self.getName())
            # 设置Event对象内部信号标志
            event.set()
# 生成Event对象
event = threading.Event()
# 设置Event对象内部信号标志
event.set()
t1 = []
for i in range(10):
    t = mThread(str(i))
    # 生成线程列表
    t1.append(t)
for i in t1:
    # 运行线程
    i.start()

The output results are as follows:

1
0
3
2
5
4
7
6
9
8

6. Background thread

By default, the main thread exits After that, even if the child thread does not join. Then after the main thread ends, the child thread will still continue to execute. If you want the main thread to exit, its sub-threads will also exit and no longer execute, you need to set the sub-threads as background threads. Python provides the setDeamon method.