An in-depth analysis of the garbage collection mechanism in Python

Thanks to Python's automatic garbage collection mechanism, there is no need to manually release objects when creating them in Python. This is very developer friendly and frees developers from having to worry about low-level memory management. But if you don’t understand its garbage collection mechanism, the Python code you write will often be very inefficient.

There are many garbage collection algorithms, the main ones are: Reference counting, Mark-clearance, Generational collection, etc.

In python, the garbage collection algorithm is based on reference counting, mark-clearance and generational collection Two mechanisms are supplemented.

1 Reference Counting

1.1 Principle of Reference Counting Algorithm

The reference counting principle is relatively simple:

• Each object has an integer reference count attribute. Used to record the number of times an object is referenced.

• For example, object A, if an object references A, then the reference count of A is +1.

• When the reference is deleted, the reference count of A is -1.

• When the reference count of A is 0, it means that object A can no longer be used and will be recycled directly.

In Python, you can get the reference counter of the specified object through the getrefcount function of the sys module Let’s look at the value of , using a practical example.

import sys

class A():
    def __init__(self):
        pass
        
a = A()
print(sys.getrefcount(a))

Run the above code, you can get the output result as 2.

1.2 Counter increase and decrease conditions

We saw above that create an A object and assign the object to the a variable After that, the reference counter value of the object is 2. So when will the counter be +1, and when will the counter be -1?

1.2.1 Conditions for reference count +1

• The object is created, such as A().
• The object is referenced, such as a=A().
• Objects are used as parameters of functions, such as func(a). The
• object is used as an element of the container, such as arr=[a,a].

1.2.2 Conditions for reference count -1

• The object is explicitly destroyed, such as del a.
• Variables are reassigned to new objects, for example a=0. The
• object leaves its scope, such as when the func function completes execution, the local variables in the func function (global variables will not).
• The container in which the object is located is destroyed, or the object is deleted from the container.

1.2.3 Code practice

In order to better understand the increase and decrease of the counter, we run the actual code and see it clearly at a glance.

import sys
 
class A():

    def __init__(self):
        pass
 
print("创建对象 0 + 1 =", sys.getrefcount(A()))

a = A()
print("创建对象并赋值 0 + 2 =", sys.getrefcount(a))

b = a
c = a
print("赋给2个变量 2 + 2 =", sys.getrefcount(a))

b = None
print("变量重新赋值 4 - 1 =", sys.getrefcount(a))

del c
print("del对象 3 - 1 =", sys.getrefcount(a))

d = [a, a, a]
print("3次加入列表 2 + 3 =", sys.getrefcount(a))


def func(c):
    print('传入函数 1 + 2 = ', sys.getrefcount(c))
func(A())

The output results are as follows:

创建对象 0 + 1 = 1
创建对象并赋值 0 + 2 = 2
赋给2个变量 2 + 2 = 4
变量重新赋值 4 - 1 = 3
del对象 3 - 1 = 2
3次加入列表 2 + 3 = 5
传入函数 1 + 2 =  3

1.3 Advantages and Disadvantages of Reference Counting

##1.3.1 Advantages of Reference Counting

Efficient and simple logic, just add and subtract the counter according to the rules.
• real-time. Once the object's counter reaches zero, it means that the object can never be used again, and there is no need to wait for a specific time to release the memory directly.

1.3.2 Disadvantages of reference counting

Needs to allocate reference counting space for the object, which increases memory consumption.
• When the object that needs to be released is relatively large, such as a dictionary object, all referenced objects need to be called in a loop and nested, which may take a long time.
• Circular reference.
• This is the fatal flaw of reference counting. Reference counting has no solution, so other garbage collection algorithms must be used to supplement it.

2 Mark-clear

As mentioned in the previous section, the reference counting algorithm cannot Solve the problem of circular reference. The object of circular reference will cause our counter to never equal

0, causing the problem of being unable to be recycled.

Mark-clear algorithm is mainly used for potential circular reference problems. The algorithm is divided into 2 steps:

• Marking stage. Treat all objects as nodes of the graph, and construct the graph structure based on the reference relationships of the objects. All objects are traversed from the root node of the graph, and all visited objects are marked to indicate that the objects are "reachable".

• Clear phase. Traverse all objects, and if an object is found not marked as "reachable", it will be recycled.

Explain with specific code examples:

class A():
    def __init__(self):
        self.obj = None
 
def func():
    a = A()
    b = A()
    c = A()
    d = A()

    a.obj = b
    b.obj = a
    return [c, d]

e = func()

In the above code, a and b refer to each other, and e refers to c and d. The entire reference relationship is shown in the figure below:

如果采用引用计数器算法，那么a和b两个对象将无法被回收。而采用标记清除法，从根节点（即e对象）开始遍历，c、d、e三个对象都会被标记为可达，而a和b无法被标记。因此a和b会被回收。

这是读者可能会有疑问，为什么确定根节点是e，而不会是a、b、c、d呢？这里就有讲究了，什么样的对象会被看成是根节点呢？一般而言，根节点的选取包括（但不限于）如下几种：

• 当前栈帧中的本地变量表中引用的对象，如各个线程被调用的方法堆栈中使用到的参数、 局部变量、 临时变量等。
• 全局静态变量
• ...

3 分代收集

3.1 分代收集原理

在执行垃圾回收过程中，程序会被暂停，即stop-the-world。这里很好理解：你妈妈在打扫房间的时候，肯定不允许你在房间内到处丢垃圾，要不然永远也无法打扫干净。

为了减少程序的暂停时间，采用分代回收(Generational Collection)降低垃圾收集耗时。

分代回收基于这样的法则：

• 接大部分的对象生命周期短，大部分对象都是朝生夕灭。

• 经历越多次数的垃圾收集且活下来的对象，说明该对象越不可能是垃圾，应该越少去收集。

Python中，对象一共有3种世代：G0,G1,G2。

• 对象刚创建时为G0。

• 如果在一轮GC扫描中存活下来，则移至G1，处于G1的对象被扫描次数会减少。

• 如果再次在扫描中活下来，则进入G2，处于G1的对象被扫描次数将会更少。

3.2 触发GC时机

当某世代中分配的对象数量与被释放的对象之差达到某个阈值的时，将触发对该代的扫描。当某世代触发扫描时，比该世代年轻的世代也会触发扫描。

那么这个阈值是多少呢？我们可以通过代码查看或者修改，示例代码如下

import gc
threshold = gc.get_threshold()
print("各世代的阈值:", threshold)

# 设置各世代阈值
# gc.set_threshold(threshold0[, threshold1[, threshold2]])
gc.set_threshold(800, 20, 20)

输出结果如下：

各世代的阈值: (700, 10, 10)

原文地址：https://juejin.cn/post/7119018622906957854

作者：SuperHua1001

【相关推荐：Python3视频教程 】

The above is the detailed content of An in-depth analysis of the garbage collection mechanism in Python. For more information, please follow other related articles on the PHP Chinese website!