A brief analysis of the use of Python yield

You may have heard that a function with yield is called a generator in Python. What is a generator?

Let’s put aside generators first and use a common programming topic to demonstrate the concept of yield.

　How to generate the Fibonacci sequence

　The Fibonacci sequence is a very simple recursive sequence. Except for the first and second numbers, any number can be obtained by adding the first two numbers. . Using a computer program to output the first N numbers of the Fibonacci sequence is a very simple problem. Many beginners can easily write the following function:

　List 1. Simply output the first N numbers of the Fibonacci sequence

deffab(max):
   n, a, b =0, 0, 1
   whilen < max:
       printb
       a, b =b, a +b
       n =n +1

　 Executing fab(5), we can get the following output:

>>> fab(5)

　 There is no problem with the result, but experienced developers will point out that using print to print numbers directly in the fab function will cause the function to The reusability is poor because the fab function returns None and other functions cannot obtain the sequence generated by this function.

　To improve the reusability of the fab function, it is best not to print out the sequence directly, but to return a List. The following is the rewritten second version of the fab function:

List 2. Output the first N numbers of the Fibonacci sequence Second version

deffab(max):
   n, a, b =0, 0, 1
   L =[]
   whilen < max:
       L.append(b)
       a, b =b, a +b
       n =n +1
   returnL

You can use the following method to print out the List returned by the fab function:

>>> forn infab(5):

... printn

...

　The rewritten fab function can meet the reusability requirements by returning a List, but more experienced developers will point out that the function is running The memory occupied will increase as the parameter max increases. If you want to control the memory usage, it is best not to use List

　 to save intermediate results, but to iterate through iterable objects. For example, in Python2. ): pass

　 will not generate a List of 1000 elements, but will return the next value in each iteration, occupying very little memory space. Because xrange does not return a List, but an iterable object.

　Using iterable, we can rewrite the fab function into a class that supports iterable. The following is the third version of Fab:

　Listing 4. The third version

classFab(object):
    
   def__init__(self, max):
       self.max=max
       self.n, self.a, self.b =0, 0, 1
    
   def__iter__(self):
       returnself
    
   defnext(self):
       ifself.n < self.max:
           r =self.b
           self.a, self.b =self.b, self.a +self.b
           self.n =self.n +1
           returnr
       raiseStopIteration()

　The Fab class continuously returns the next one in the sequence through next() Number, the memory usage is always constant:

>>> forn inFab(5):

... printn

...

However, the code of this version rewritten using class is far from the first version of fab Functions come in simple terms. If we want to keep the simplicity of the first version of the fab function while getting iterable effects, yield comes in handy:

　 Listing 5. The fourth version of fab using yield

deffab(max):
    n, a, b =0, 0, 1
    whilen < max:
        yieldb
        # print b
        a, b =b, a +b
        n =n +1

The fourth version of fab and Compared with the first version, just changing print b to yield b achieves the iterable effect while maintaining simplicity.

　 Calling the fourth version of fab is exactly the same as the second version of fab:

>>> forn infab(5):

... printn

...

　 Simply put, the function of yield is to A function becomes a generator. The function with yield is no longer an ordinary function. The Python interpreter will treat it as a generator. Calling fab(5) will not execute the fab function, but return an iterable object! When the for loop is executed, each loop will execute the code inside the fab function. When it reaches yield b, the fab function returns an iteration value. In the next iteration, the code continues to execute from the next statement of yield b, and the function The local variable looks exactly the same as it did before the last interruption, so the function continues execution until yield is encountered again.

　You can also manually call the next() method of fab(5) (because fab(5) is a generator object, which has a next() method), so that we can see the execution process of fab more clearly:

　List 6. Execution process

>>> f =fab(5)

>>> f.next()

>>> f.next()

>>> f.next()

> >> f.next()

>>> f.next()

>>> f.next()

Traceback (most recent call last):

File"", line 1, in StopIteration

When the function execution ends, the generator automatically throws a StopIteration exception, indicating that the iteration is completed. In the for loop, there is no need to handle the StopIteration exception, and the loop will end normally.

　We can draw the following conclusion:

　A function with yield is a generator. It is different from an ordinary function. Generating a generator looks like a function call, but will not execute any function code until next( is called on it. ) (next() will be automatically called in the for loop) before execution starts. Although the execution flow is still executed according to the flow of the function, every time a yield statement is executed, it will be interrupted and an iteration value will be returned. The next execution will continue from the next statement of yield. It looks like a function is interrupted several times by yield during normal execution, and each interruption returns the current iteration value through yield.

　　yield 的好处是显而易见的，把一个函数改写为一个 generator 就获得了迭代能力，比起用类的实例保存状态来计算下一个 next() 的值，不仅代码简洁，而且执行流程异常清晰。

　　如何判断一个函数是否是一个特殊的 generator 函数？可以利用 isgeneratorfunction 判断：

　　清单 7. 使用 isgeneratorfunction 判断

>>> frominspect importisgeneratorfunction

>>> isgeneratorfunction(fab)

True

　　要注意区分 fab 和 fab(5)，fab 是一个 generator function，而 fab(5) 是调用 fab 返回的一个 generator，好比类的定义和类的实例的区别：

　　清单 8. 类的定义和类的实例

>>> importtypes

>>> isinstance(fab, types.GeneratorType)

False

>>> isinstance(fab(5), types.GeneratorType)

True

　　fab 是无法迭代的，而 fab(5) 是可迭代的：

>>> fromcollections importIterable

>>> isinstance(fab, Iterable)

False

>>> isinstance(fab(5), Iterable)

True

　　每次调用 fab 函数都会生成一个新的 generator 实例，各实例互不影响：

>>> f1 =fab(3)

>>> f2 =fab(5)

>>> print'f1:', f1.next()

f1: 1

>>> print'f2:', f2.next()

f2: 1

>>> print'f1:', f1.next()

f1: 1

>>> print'f2:', f2.next()

f2: 1

>>> print'f1:', f1.next()

f1: 2

>>> print'f2:', f2.next()

f2: 2

>>> print'f2:', f2.next()

f2: 3

>>> print'f2:', f2.next()

f2: 5

　　return 的作用

　　在一个 generator function 中，如果没有 return，则默认执行至函数完毕，如果在执行过程中 return，则直接抛出 StopIteration 终止迭代。

　　另一个例子

　　另一个 yield 的例子来源于文件读取。如果直接对文件对象调用 read() 方法，会导致不可预测的内存占用。好的方法是利用固定长度的缓冲区来不断读取文件内容。通过 yield，我们不再需要编写读文件的迭代类，就可以轻松实现文件读取：

　　清单 9. 另一个 yield 的例子

defread_file(fpath):
   BLOCK_SIZE =1024
   with open(fpath, 'rb') as f:
       whileTrue:
           block =f.read(BLOCK_SIZE)
           ifblock:
               yieldblock
           else:
               return

　　以上仅仅简单介绍了 yield 的基本概念和用法，yield 在 Python 3 中还有更强大的用法，我们会在后续文章中讨论。

　　注：本文的代码均在 Python 2.7 中调试通过