Home > Backend Development > Python Tutorial > python greenlet implementation mechanism

python greenlet implementation mechanism

巴扎黑
Release: 2016-12-09 14:45:47
Original
1225 people have browsed it

Recently, I use Python to develop web programs, and I have always used fastcgi mode. Then multiple threads are started in each process to process requests. One problem here is that it needs to ensure that the response time of each request is very short, otherwise it will only take multiple requests. A few slowdowns will cause the server to refuse service because no thread can respond to the request. We usually conduct performance tests when our services come online, so there is not much of a problem under normal circumstances. But it is impossible to test all scenarios. Once it occurs, It will make the user wait for a long time without responding. Some parts are unavailable, which leads to all unavailability. Later, I switched to coroutine and greenlet under python. So I have a simple understanding of its implementation mechanism.
Each greenlet is just a python in the heap object (PyGreenlet). So it is no problem for you to create millions or even tens of millions of greenlets for a process.

typedef struct _greenlet {
PyObject_HEAD
char* stack_start;
char* stack_stop;
char* stack_copy;
intptr_t stack_saved;
struct _greenlet* stack_prev;
struct _greenlet* parent;
PyObject* run_info;
struct _frame* top_frame;
int recursion_depth;
PyObject* weakreflist;
PyObject* exc_type;
PyObject* exc_value;
PyObject* exc_traceback;
PyObject* dict;
} PyGreenlet;
Copy after login

Each greenlet is actually a function, and saves the context when this function is executed. For a function, the context is its stack.. All greenlets in the same process share a common user stack allocated by the operating system. Therefore, only greenlets with non-conflicting stack data can use this global stack at the same time. Greenlets save their stacks through stack_stop and stack_start. At the bottom and top of the stack, if the stack_stop of the greenlet to be executed overlaps with the greenlet currently in the stack, the stack data of these overlapping greenlets must be temporarily saved to the heap. The saved location is determined by stack_copy and stack_saved. Record so that the positions of stack_stop and stack_start in the stack can be copied from the heap back to the stack during recovery. Otherwise, the stack data will be destroyed. Therefore, these greenlets created by the application are constantly copying data to the heap or from The heap is copied to the stack to achieve concurrency. It is really comfortable to use coroutine for IO-type applications.

The following is a simple stack space model of greenlet (from greenlet.c)

A PyGreenlet is a range of C stack addresses that must be
saved and restored in such a way that the full range of the
stack contains valid data when we switch to it.
Stack layout for a greenlet:
               |     ^^^       |
               |  older data   |
               |               |
  stack_stop . |_______________|
        .      |               |
        .      | greenlet data |
        .      |   in stack    |
        .    * |_______________| . .  _____________  stack_copy + stack_saved
        .      |               |     |             |
        .      |     data      |     |greenlet data|
        .      |   unrelated   |     |    saved    |
        .      |      to       |     |   in heap   |
 stack_start . |     this      | . . |_____________| stack_copy
               |   greenlet    |
               |               |
               |  newer data   |
               |     vvv       |
Copy after login

The following is a simple paragraph greenlet code.

from greenlet import greenlet
def test1():
    print 12
    gr2.switch()
    print 34
def test2():
    print 56
    gr1.switch()
    print 78
gr1 = greenlet(test1)
gr2 = greenlet(test2)
gr1.switch()
Copy after login


Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template