Python yield usage analysis

You may have heard that a function with yield is called a generator in Python. What is a generator?

Let’s put aside generators first and use a common programming problem to demonstrate the concept of yield.

　How to generate Fibonacci numbers

The Fibonacci sequence is a very simple recursive sequence. Except for the first and second numbers, any number can be obtained by adding the first two numbers. Using a computer program to output the first N numbers of the Fibonacci sequence is a very simple problem. Many beginners can easily write the following function:

　List 1. Simple output of the first N numbers of the Fibonacci sequence

 def fab(max): 
    n, a, b = 0, 0, 1 
    while n < max: 
        print b 
        a, b = b, a + b 
        n = n + 1

Executing fab(5), we can get the following output:

 >>> fab(5) 
 1 
 1 
 2 
 3 
 5

There is no problem with the result, but experienced developers will point out that using print to print numbers directly in the fab function will make the function less reusable, because the fab function returns None and other functions cannot obtain the sequence generated by the function.

To improve the reusability of the fab function, it is best not to print out the sequence directly, but to return a List. The following is the rewritten second version of the fab function:

　List 2. Output the first N numbers of the Fibonacci sequence second version

 def fab(max): 
    n, a, b = 0, 0, 1 
    L = [] 
    while n < max: 
        L.append(b) 
        a, b = b, a + b 
        n = n + 1 
    return L

You can use the following method to print out the List returned by the fab function:

 >>> for n in fab(5): 
 ...     print n 
 ... 
 1 
 1 
 2 
 3 
 5

The rewritten fab function can meet the reusability requirements by returning a List, but more experienced developers will point out that the memory occupied by this function during operation will increase as the parameter max increases. If you want to control the memory Occupied, it is best not to use List

To save intermediate results, iterate through iterable objects. For example, in Python2.x, code:

　Listing 3. Iterating through iterable objects

 for i in range(1000): pass

will result in generating a List of 1000 elements, and the code:

 for i in xrange(1000): pass

will not generate a List of 1000 elements, but will return the next value in each iteration, occupying very little memory space. Because xrange does not return a List, but an iterable object.

Using iterable we can rewrite the fab function into a class that supports iterable. The following is the third version of Fab:

　List 4. The third version

 class Fab(object): 

    def __init__(self, max): 
        self.max = max 
        self.n, self.a, self.b = 0, 0, 1 

    def __iter__(self): 
        return self 

    def next(self): 
        if self.n < self.max: 
            r = self.b 
            self.a, self.b = self.b, self.a + self.b 
            self.n = self.n + 1 
            return r 
        raise StopIteration()

The Fab class continuously returns the next number in the sequence through next(), and the memory usage is always constant:

 >>> for n in Fab(5): 
 ...     print n 
 ... 
 1 
 1 
 2 
 3 
 5

However, the code of this version rewritten using class is far less concise than the first version of the fab function. Yield comes in handy if we want to keep the simplicity of the first version of the fab function while getting the effect of iterable:

Listing 5. Fourth version using yield

 def fab(max): 
    n, a, b = 0, 0, 1 
    while n < max: 
        yield b 
        # print b 
        a, b = b, a + b 
        n = n + 1 

&#39;&#39;&#39;

Compared with the first version, the fourth version of fab only changed print b to yield b, which achieved the effect of iterable while maintaining simplicity.

Calling the fourth version of fab is exactly the same as the second version of fab:

 >>> for n in fab(5): 
 ...     print n 
 ... 
 1 
 1 
 2 
 3 
 5

Simply put, the function of yield is to turn a function into a generator. The function with yield is no longer an ordinary function. The Python interpreter will treat it as a generator. Calling fab(5) will not execute the fab function. Instead, it returns an iterable object! When the for loop is executed, each loop will execute the code inside the fab function. When it reaches yield b, the fab function returns an iteration value. In the next iteration, the code continues to execute from the next statement of yield b, and the function The local variable looks exactly the same as it did before the last interruption, so the function continues execution until yield is encountered again.

You can also manually call the next() method of fab(5) (because fab(5) is a generator object, which has a next() method), so that we can see the execution process of fab more clearly:

List 6. Execution process

 >>> f = fab(5) 
 >>> f.next() 
 1 
 >>> f.next() 
 1 
 >>> f.next() 
 2 
 >>> f.next() 
 3 
 >>> f.next() 
 5 
 >>> f.next() 
 Traceback (most recent call last): 
  File "<stdin>", line 1, in <module> 
 StopIteration

When the function execution ends, the generator automatically throws a StopIteration exception, indicating that the iteration is completed. In the for loop, there is no need to handle the StopIteration exception, and the loop will end normally.

​We can draw the following conclusions:

A function with yield is a generator. It is different from an ordinary function. Generating a generator looks like a function call, but will not execute any function code until next() is called on it (next( will be called automatically in a for loop) )) before starting execution. Although the execution flow is still executed according to the flow of the function, every time a yield statement is executed, it will be interrupted and an iteration value will be returned. The next execution will continue from the next statement of yield. It looks like a function is interrupted several times by yield during normal execution, and each interruption returns the current iteration value through yield.

The benefits of yield are obvious. By rewriting a function as a generator, you gain the ability to iterate. Compared with using an instance of a class to save the state to calculate the next() value, not only is the code concise, but the execution process is extremely clear.

　　如何判断一个函数是否是一个特殊的 generator 函数？可以利用 isgeneratorfunction 判断：

　　清单 7. 使用 isgeneratorfunction 判断

 >>> from inspect import isgeneratorfunction 
 >>> isgeneratorfunction(fab) 
 True

　　要注意区分 fab 和 fab(5)，fab 是一个 generator function，而 fab(5) 是调用 fab 返回的一个 generator，好比类的定义和类的实例的区别：

　　清单 8. 类的定义和类的实例

 >>> import types 
 >>> isinstance(fab, types.GeneratorType) 
 False 
 >>> isinstance(fab(5), types.GeneratorType) 
 True

　　fab 是无法迭代的，而 fab(5) 是可迭代的：

 >>> from collections import Iterable 
 >>> isinstance(fab, Iterable) 
 False 
 >>> isinstance(fab(5), Iterable) 
 True

　　每次调用 fab 函数都会生成一个新的 generator 实例，各实例互不影响：

 >>> f1 = fab(3) 
 >>> f2 = fab(5) 
 >>> print 'f1:', f1.next() 
 f1: 1 
 >>> print 'f2:', f2.next() 
 f2: 1 
 >>> print 'f1:', f1.next() 
 f1: 1 
 >>> print 'f2:', f2.next() 
 f2: 1 
 >>> print 'f1:', f1.next() 
 f1: 2 
 >>> print 'f2:', f2.next() 
 f2: 2 
 >>> print 'f2:', f2.next() 
 f2: 3 
 >>> print 'f2:', f2.next() 
 f2: 5

　　return 的作用

　　在一个 generator function 中，如果没有 return，则默认执行至函数完毕，如果在执行过程中 return，则直接抛出 StopIteration 终止迭代。

　　另一个例子

　　另一个 yield 的例子来源于文件读取。如果直接对文件对象调用 read() 方法，会导致不可预测的内存占用。好的方法是利用固定长度的缓冲区来不断读取文件内容。通过 yield，我们不再需要编写读文件的迭代类，就可以轻松实现文件读取：

　　清单 9. 另一个 yield 的例子

 def read_file(fpath): 
    BLOCK_SIZE = 1024 
    with open(fpath, 'rb') as f: 
        while True: 
            block = f.read(BLOCK_SIZE) 
            if block: 
                yield block 
            else: 
                return

　　以上仅仅简单介绍了 yield 的基本概念和用法，yield 在 Python 3 中还有更强大的用法，我们会在后续文章中讨论。

　　注：本文的代码均在 Python 2.7 中调试通过

The above is the detailed content of Python yield usage analysis. For more information, please follow other related articles on the PHP Chinese website!