What are the techniques to improve Python's execution efficiency?

Before starting, you can develop a python decorator that counts function running time, which will be used later for time statistics after we use various python techniques.

# 导入时间提取的time模块
from time import time

import dis


def compute_time(func_):
    '''
    计算函数的运行时间
    '''

    def func_time(*arg, **kw):
        t1 = time()
        result = func_(*arg, **kw)
        t2 = time()
        print(f"{func_.__name__: >10} : {t2 - t1:.6f} 秒")
        return result

    return func_time

We have developed the above compute_time time calculation function. We can develop a hello_world function to test whether it is used normally.

@compute_time
def hello_world():
    print("hello_world!")


hello_world()

# hello_world!
# hello_world : 0.000000 秒

Through the test of hello_world function, it is proved that our time decorator compute_time can normally calculate the running time of the function.

Next, we begin to formally introduce the following five ways to improve the running speed of python and provide time running results.

1. Reasonable use of standard or non-standard libraries

You must not underestimate python’s standard or non-standard libraries during the development process. To be honest, we sometimes write the same business code blocks ourselves. It is true that there is no perfection like the big guys.

For example, in the following business, we need to convert the values ​​in a python list into strings. First, look at how the following code block is written.

# 初始化一个list列表
list_ = ['a', 'b', 'c'] * 10000


@compute_time
def func_1(list_=None):
    '''
    列表元素转字符串函数
    '''
    str_ = ''
    for s in list_:
        str_ = str_ + s
    return str_


func_1(list_)


# func_1 : 0.001999 秒

Using the traditional method written by myself to convert the execution of the func_1 function above is more complicated and takes 0.001999 seconds.

@compute_time
def func_2(list_=None):
    '''
    列表元素转字符串
    '''
    return ''.join(list_)


func_2(list_)

# func_2 : 0.000000 秒

Compared with the running time of func_1 function, the running time of func_2 is almost negligible, and the six-digit decimal cannot see the change at all.

2. Reduce the use of loops

In fact, we have discovered from the normal development process that serializable data processing methods using list derivation, iteration, etc. are more convenient than for loops. , efficient.

Below we can also use an example to illustrate the problem. For example, we need to select a number in a list that is divisible by 2.

# 初始化循环次数n
n = 100000


@compute_time
def func_3(n=None):
    list_ = []
    for m in range(n):
        if m % 2 == 0:
            list_.append(m)
    return list_


@compute_time
def func_4(n=None):
    return [m for m in range(n) if m % 2 == 0]


func_3(n)

func_4(n)

# func_3 : 0.004986 秒
# func_4 : 0.003014 秒

Through the comparison of func_3 function and func_4 function, first of all, the method of func_4 is much simpler than func_3.

And in terms of time, func_4 uses list derivation to run 1/4 faster than the ordinary for loop.

3. Pay attention to repeated code running

Regarding the repeated running of code, we can all experience this in our usual development methods, that is, it can be run once as a public code block.

You can add code blocks that can be used publicly into the loop, which will only affect the execution efficiency of the code blocks.

For example, we need to use python's re module to search for certain elements in the string. Here are two ways to compare the time results.

# 导入正则表达式匹配模块
import re


@compute_time
def func_5(str_=None):
    for s in str_:
        result = re.search(r'a*[a-z]?c', s)


@compute_time
def func_6(str_=None):
    repx = re.compile(r'a*[a-z]?c')
    for s in str_:
        result = repx.search(s)


func_5('abcdefg1234oks' * 1000)

func_6('abcdefg1234oks' * 1000)

# func_5 : 0.006999 秒
# func_6 : 0.002000 秒

Comparing the business implementation methods of func_5 and func_6, we put the compile regular matching object of the re module directly into the outer layer of the for loop, and the running time was directly reduced by more than 3 times.

This is because using search to match regular objects directly in the loop will continuously create regular matching objects in the loop, which
increases the processing burden of the for loop and causes the speed to slow down.

4. Reduce the use of global variables

When explaining this point, we must understand that global variables always exist and will not disappear during the running of the program.

Too many global variables will cause too much memory to be occupied during operation. It will be more efficient to use local variables compared to global variables.

Below we use examples of two methods to compare the running time of global variables and local variables.

mes_1 = 'ss1'

mes_2 = 'ss2'

mes_3 = 'ss3'


@compute_time
def func_7():
    result = mes_1 + mes_2 + mes_3
    return result


@compute_time
def func_8():
    me_1 = 'ss1'
    me_2 = 'ss2'
    me_3 = 'ss3'
    result = me_1 + me_2 + me_3
    return result


func_7()

func_8()


# func_7 : 0.000997 秒
# func_8 : 0.000000 秒

We have done an ordinary addition calculation above to illustrate the problem. The way func_8 function uses local variables is indeed faster.

5. Use reasonable data structures

In most Python development processes, many people must use lists to process data for convenience.

Python has four built-in data structures: lists, tuples, sets, and dictionaries. Using appropriate data structures to process data in appropriate business scenarios can also improve calculation execution efficiency.

For example: Below we will extract the value at the corresponding index position from a list list and tuple tuple.

@compute_time
def func_9():
    data = ['a', 'b', 'c', 'd', 'e', 'f', 'g', 'h']
    print(data[3])


@compute_time
def func_10():
    data = ('a', 'b', 'c', 'd', 'e', 'f', 'g', 'h')
    print(data[3])

func_9()

func_10()

# func_9 : 0.000000 秒
# func_10 : 0.000000 秒

By executing the func_9 and func_10 functions, we found that the difference in time between the two is not big, at least the result cannot be distinguished within six decimal places.

print('func_9汇编产生的机器码：')
dis.dis(func_9)

print('func_10汇编产生的机器码：')
dis.dis(func_10)

Finally, we checked the assembly machine codes of func_9 and func_10 respectively, and found that obviously list list processing generates more machine codes.

# func_9汇编产生的机器码：
#  30           0 LOAD_GLOBAL              0 (time)
#               2 CALL_FUNCTION            0
#               4 STORE_FAST               2 (t1)
#
#  31           6 LOAD_DEREF               0 (func_)
#               8 LOAD_FAST                0 (arg)
#              10 LOAD_FAST                1 (kw)
#              12 CALL_FUNCTION_EX         1
#              14 STORE_FAST               3 (result)
#
#  32          16 LOAD_GLOBAL              0 (time)
#              18 CALL_FUNCTION            0
#              20 STORE_FAST               4 (t2)
#
#  33          22 LOAD_GLOBAL              1 (print)
#              24 LOAD_DEREF               0 (func_)
#              26 LOAD_ATTR                2 (__name__)
#              28 LOAD_CONST               1 (' >10')
#              30 FORMAT_VALUE             4 (with format)
#              32 LOAD_CONST               2 (' : ')
#              34 LOAD_FAST                4 (t2)
#              36 LOAD_FAST                2 (t1)
#              38 BINARY_SUBTRACT
#              40 LOAD_CONST               3 ('.6f')
#              42 FORMAT_VALUE             4 (with format)
#              44 LOAD_CONST               4 (' 秒')
#              46 BUILD_STRING             4
#              48 CALL_FUNCTION            1
#              50 POP_TOP
#
#  34          52 LOAD_FAST                3 (result)
#              54 RETURN_VALUE
# func_10汇编产生的机器码：
#  30           0 LOAD_GLOBAL              0 (time)
#               2 CALL_FUNCTION            0
#               4 STORE_FAST               2 (t1)
#
#  31           6 LOAD_DEREF               0 (func_)
#               8 LOAD_FAST                0 (arg)
#              10 LOAD_FAST                1 (kw)
#              12 CALL_FUNCTION_EX         1
#              14 STORE_FAST               3 (result)

The above is the detailed content of What are the techniques to improve Python's execution efficiency?. For more information, please follow other related articles on the PHP Chinese website!