Table of Contents
Concept:
Generator implements coroutine principle
Introduction of greenlet
gevent coroutine
gevent coroutine communication
Home Backend Development Python Tutorial Deep understanding of coroutine functions in python

Deep understanding of coroutine functions in python

Apr 14, 2018 am 11:16 AM
python function coroutine

The content of this article is to share with you an in-depth understanding of the coroutine function in python, which has a certain reference value. Friends in need can refer to it

Concept:

According to the definition given by Wikipedia, "Coroutines are computer program components that generate subroutines for non-preemptive multitasking. Coroutines allow different entry points to pause or start program execution at different locations." From a technical perspective, "a coroutine is a function that you can pause execution of." If you understand it as "just like a generator", then you're thinking right.

Coroutine, also known as micro-thread, looks like a subroutine, but it is different from a subroutine. During the execution process, it can interrupt the current subroutine. After the program executes other subroutines, it returns to execute the previous subroutine, but its related information is still the same as before.

Coroutines are different from threads. Threads are preemptive scheduling, while coroutines are collaborative scheduling. Coroutines need to do their own scheduling.
Subroutine calls always have one entrance and one return, and the calling sequence is clear. The calling of coroutines is different from subroutines. Coroutines also look like subprograms, but during execution, they can be interrupted inside the subprogram, then switch to executing other subprograms, and then return to continue execution at the appropriate time.

Advantages of coroutines:

  • The advantage of coroutines is extremely high execution efficiency. Because subroutine switching is not thread switching, but is controlled by the program itself, there is no overhead of thread switching. Compared with multi-threading, the greater the number of threads, the more obvious the performance advantages of coroutines. It is very suitable for performing coroutine multitasking.

  • Coroutines have no thread safety issues. A process can have multiple coroutines at the same time, but only one coroutine is active, and the activation and dormancy of coroutines are controlled by programmers through programming, not by the operating system.

Generator implements coroutine principle

Example:

def func(n):
    index=0
    if index<=n:
        c=yield 1
        print("task------{}".format(c))
        index+=1f=func(3)
n=next(f)
print(n)try:
    n=f.send(5)#程序就直接结束了
    print("n是{}".format(n))except StopIteration as e:    pass
Copy after login
输出打印:1task------5
Copy after login

Explanation:

  • Obviously func is a generator, and the send method has a parameter that specifies the return value of the last suspended yield statement.

  • send requires exception handling.

  • In general, the only difference between the send method and the next method is that when executing the send method, the return value of the last pending yield statement will first be set through parameters to achieve Interaction with generator methods. However, it should be noted that before a generator object executes the next method, since no yield statement is suspended, an error will be reported when executing the send method.

  • When the parameter of the send method is None, it is completely equivalent to the next method.

The generator implements the producer and consumer patterns:

def cunsumer():
    while True:
        n=yield 3
        if not n:            return
        print(&#39;cunsumer{}&#39;.format(n))def product(c):
    c.send(None)
    n=0
    while n<5:
        n=n+1
        r=c.send(n)
        print("product{}".format(r))
    c.close()
c=cunsumer()
product(c)
Copy after login
打印:
cunsumer1
product3
cunsumer2
product3
cunsumer3
product3
cunsumer4
product3
cunsumer5
product3
Copy after login

Explanation:

Execute first in the producer The purpose of c.send(None) is to let the consumer hang up first, and then use send to pass the value. The first time 1 is passed, the consumer prints 1, and the producer prints r which is the value after the consumer's yield.

Introduction of greenlet

Although CPython (standard Python) can implement coroutines through generators, it is not very convenient to use.

At the same time, Stackless Python, a derivative of Python, implements native coroutines, which is more convenient to use.

So, everyone began to take out the coroutine code in Stackless and make it into a CPython expansion package.

This is the origin of greenlet, so greenlet is a C extension library that implements native coroutines at the bottom.

Code description:

from greenlet import greenletimport randomimport timedef Producer():
    while True:
        item = random.randint(0,10)
        print("生产了{}".format(item))
        c.switch(item)#切换到消费者,并将item传入消费者
        time.sleep(1)def consumer():
    print(&#39;我先执行&#39;)    #p.switch()
    while True:
        item = p.switch()#切换到生产者,并且等待生产者传入item
        print(&#39;消费了{}&#39;.format(item))
c = greenlet(consumer)#将一个普通函数变成一个协程p = greenlet(Producer)
c.switch()#让消费者先进入暂停状态(只有恢复了才能接收数据)
Copy after login

Value of greenlet:

  • High-performance native coroutine

  • Explicit switching with clearer semantics

  • Wrap functions directly into coroutines and maintain the original code style

gevent coroutine

Although we have a callback programming model based on epoll, it is difficult to use.

Even though we can make complex encapsulation with generator coroutine to simplify programming difficulty.
But there is still a big problem: encapsulation is difficult, and the existing code must be almost completely rewritten

gevent, by encapsulating the two libraries libev (based on epoll) and greenlet.
Help us encapsulate it and allow us to use coroutines in a thread-like manner.

So that we can make full use of the power of epoll and coroutines without rewriting the original code.

Code diagram:

from gevent import monkey;monkey.patch_all()#会把python标准库当中一些阻塞操作变成非阻塞import geventdef test1():
    print("11")
    gevent.sleep(4)#模拟爬虫请求阻塞
    print("33")def test2():
    print("22")
    gevent.sleep(4)
    print("44")
gevent.joinall([gevent.spawn(test1),gevent.spawn(test2)])#joinall 阻塞当前协程,执行给定的greenlet#spawn 启动协程,参数就是函数的名字
Copy after login

The value of gevent:

Switch to another coroutine to continue execution when it encounters blocking !

  • Use epoll-based libev to avoid blocking.

  • Use efficient coroutines based on gevent to switch execution.

  • Only switches when encountering blocking. There is no round-robin overhead or thread overhead.

gevent implements concurrent server

from gevent import monkey;monkey.patch_all()  #建议放在首行,会把python标准库当中一些阻塞操作变成非阻塞import geventimport socket


server=socket.socket()
server.bind((&#39;&#39;,6666))
server.listen(5)
print("开始监听")def readable(con,addr):
    print("客户端{}接入".format(addr))    while True:
        data=con.recv(1024)        if data:
            print(data)        else:
            con.close()            breakwhile True:
    con,addr=server.accept()
    gevent.spawn(readable,con,addr)#将readable函数变为协程,并且把con和addr传入其中。
Copy after login

gevent coroutine communication

gevent also has its own queue. The usage is basically the same as threading.

Producer and consumer patterns based on gevent and queue

from gevent import monkey;monkey.patch_all()import geventfrom gevent.queue import Queueimport randomdef producter(queue):
    while True:
        item=random.randint(0,99)
        print(&#39;生产了{}&#39;.format(item))
        queue.put(item)
        gevent.sleep(1)def comuser(queue):
    while True:
        item=queue.get()
        print(&#39;消费了{}&#39;.format(item))
queue=Queue()
p=gevent.spawn(producter,queue)
c=gevent.spawn(comuser,queue)
gevent.joinall([p,c])
Copy after login
打印:
生产了33消费了33生产了95消费了95生产了92消费了92...
Copy after login


相关推荐:

python中多进程+协程的使用

python中协程

Python 协程的详细用法和例子

python 协程示例

The above is the detailed content of Deep understanding of coroutine functions in python. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Choosing Between PHP and Python: A Guide Choosing Between PHP and Python: A Guide Apr 18, 2025 am 12:24 AM

PHP is suitable for web development and rapid prototyping, and Python is suitable for data science and machine learning. 1.PHP is used for dynamic web development, with simple syntax and suitable for rapid development. 2. Python has concise syntax, is suitable for multiple fields, and has a strong library ecosystem.

PHP and Python: Different Paradigms Explained PHP and Python: Different Paradigms Explained Apr 18, 2025 am 12:26 AM

PHP is mainly procedural programming, but also supports object-oriented programming (OOP); Python supports a variety of paradigms, including OOP, functional and procedural programming. PHP is suitable for web development, and Python is suitable for a variety of applications such as data analysis and machine learning.

Can vs code run in Windows 8 Can vs code run in Windows 8 Apr 15, 2025 pm 07:24 PM

VS Code can run on Windows 8, but the experience may not be great. First make sure the system has been updated to the latest patch, then download the VS Code installation package that matches the system architecture and install it as prompted. After installation, be aware that some extensions may be incompatible with Windows 8 and need to look for alternative extensions or use newer Windows systems in a virtual machine. Install the necessary extensions to check whether they work properly. Although VS Code is feasible on Windows 8, it is recommended to upgrade to a newer Windows system for a better development experience and security.

Is the vscode extension malicious? Is the vscode extension malicious? Apr 15, 2025 pm 07:57 PM

VS Code extensions pose malicious risks, such as hiding malicious code, exploiting vulnerabilities, and masturbating as legitimate extensions. Methods to identify malicious extensions include: checking publishers, reading comments, checking code, and installing with caution. Security measures also include: security awareness, good habits, regular updates and antivirus software.

Can visual studio code be used in python Can visual studio code be used in python Apr 15, 2025 pm 08:18 PM

VS Code can be used to write Python and provides many features that make it an ideal tool for developing Python applications. It allows users to: install Python extensions to get functions such as code completion, syntax highlighting, and debugging. Use the debugger to track code step by step, find and fix errors. Integrate Git for version control. Use code formatting tools to maintain code consistency. Use the Linting tool to spot potential problems ahead of time.

How to run programs in terminal vscode How to run programs in terminal vscode Apr 15, 2025 pm 06:42 PM

In VS Code, you can run the program in the terminal through the following steps: Prepare the code and open the integrated terminal to ensure that the code directory is consistent with the terminal working directory. Select the run command according to the programming language (such as Python's python your_file_name.py) to check whether it runs successfully and resolve errors. Use the debugger to improve debugging efficiency.

Python vs. JavaScript: The Learning Curve and Ease of Use Python vs. JavaScript: The Learning Curve and Ease of Use Apr 16, 2025 am 12:12 AM

Python is more suitable for beginners, with a smooth learning curve and concise syntax; JavaScript is suitable for front-end development, with a steep learning curve and flexible syntax. 1. Python syntax is intuitive and suitable for data science and back-end development. 2. JavaScript is flexible and widely used in front-end and server-side programming.

Can vscode be used for mac Can vscode be used for mac Apr 15, 2025 pm 07:36 PM

VS Code is available on Mac. It has powerful extensions, Git integration, terminal and debugger, and also offers a wealth of setup options. However, for particularly large projects or highly professional development, VS Code may have performance or functional limitations.

See all articles