Magic methods in Python-Python Tutorial-php.cn

Magic methods in python are special methods that allow you to add "magic" to a class. They are often named surrounded by two underscores.

Magic methods in Python

Python’s magic method, also known as the dunder (double underline) method. Most of the time, we use them for simple things like constructors (init), string representations (str, repr) or arithmetic operators (add/mul). In fact, there are many methods that you may not have heard of but are very useful. In this article, we will sort out these magic methods!

Iterator size

We all know the __len__ method, which can be used to implement the len() function on the container class. But what if you want to get the length of a class object that implements an iterator?

it = iter(range(100))
 print(it.__length_hint__())
 # 100
 next(it)
 print(it.__length_hint__())
 # 99
 
 a = [1, 2, 3, 4, 5]
 it = iter(a)
 print(it.__length_hint__())
 # 5
 next(it)
 print(it.__length_hint__())
 # 4
 a.append(6)
 print(it.__length_hint__())
 # 5

Copy after login

All you need to do is implement the __length_hint__ method, which is a built-in method on iterators (not generators) , as you can see above, and also supports dynamic length changes. However, true to its name, this is just a hint and is not guaranteed to be completely accurate: for list iterators, you can get accurate results, but for other iterators it is not sure. But even if it's not accurate, it can help us get the information we need, as explained in PEP 424.

length_hint must return an integer (else a TypeError is raised) or NotImplemented, and is not required to be accurate. It may return a value that is either larger or smaller than the actual size of the container. A return value of NotImplemented indicates that there is no finite length estimate. It may not return a negative value (else a ValueError is raised).

Metaprogramming

Most of the rarely seen magical methods are related to metaprogramming. Although metaprogramming may not be something we need to use every day, there are some handy tricks that can use it.

One such trick is to use __init_subclass__ as a shortcut to extend the functionality of a base class without having to deal with metaclasses:

 class Pet:
def __init_subclass__(cls, /, default_breed, **kwargs):
super().__init_subclass__(**kwargs)
cls.default_breed = default_breed
 
 class Dog(Pet, default_name="German Shepherd"):
pass

Copy after login

In the code above we add keyword arguments to the base class, the Parameters can be set when defining the subclass. A real use case might be to use this method when you want to handle the supplied parameters rather than just assigning to a property.

It seems very obscure and rarely used, but in fact you may have encountered it many times, because it is generally used when building APIs, such as in SQLAlchemy or Flask Views. arrive.

Another metaclass magic method is __call__. This method allows customization of what happens when a class instance is called:

 class CallableClass:
def __call__(self, *args, **kwargs):
print("I was called!")
 
 instance = CallableClass()
 
 instance()
 # I was called!

Copy after login

You can use this to create a class that cannot be called:

 class NoInstances(type):
def __call__(cls, *args, **kwargs):
raise TypeError("Can't create instance of this class")
 
 class SomeClass(metaclass=NoInstances):
@staticmethod
def func(x):
print('A static method')
 
 instance = SomeClass()
 # TypeError: Can't create instance of this class

Copy after login

For classes with only static methods, there is no need to create the class This method is used in the example.

Another similar scenario is the singleton pattern - a class can only have at most one instance:

 class Singleton(type):
def __init__(cls, *args, **kwargs):
cls.__instance = None
super().__init__(*args, **kwargs)
 
def __call__(cls, *args, **kwargs):
if cls.__instance is None:
cls.__instance = super().__call__(*args, **kwargs)
return cls.__instance
else:
return cls.__instance
 
 class Logger(metaclass=Singleton):
def __init__(self):
print("Creating global Logger instance")

Copy after login

Singleton class has a private __instance - if not, it will be created and Assigned, it will only be returned if it already exists.

Suppose there is a class and you want to create an instance of it without calling __init__. The __new__ method can help solve this problem:

 class Document:
def __init__(self, text):
self.text = text
 
 bare_document = Document.__new__(Document)
 print(bare_document.text)
 # AttributeError: 'Document' object has no attribute 'text'
 
 setattr(bare_document, "text", "Text of the document")

Copy after login

In some cases, we may need to bypass the usual process of creating an instance, and the above code demonstrates how to do this. Instead of calling Document(…), we call Document.__new__(Document), which creates a bare instance without calling __init__. Therefore, the attribute of the instance (text in this case) is not initialized, so we need to additionally use the setattr function to assign the value (it is also a magic method __setattr__).

Why do you do that. Because we may want to replace the constructor, such as:

 class Document:
def __init__(self, text):
self.text = text
 
@classmethod
def from_file(cls, file): # Alternative constructor
d = cls.__new__(cls)
# Do stuff...
return d

Copy after login

The from_file method is defined here, which acts as a constructor, first using __new__ to create an instance, and then configuring it without calling __init__.

The next magical method related to metaprogramming is __getattr__. This method is called when normal property access fails. This can be used to delegate access/calls to missing methods to another class:

 class String:
def __init__(self, value):
self._value = str(value)
 
def custom_operation(self):
pass
 
def __getattr__(self, name):
return getattr(self._value, name)
 
 s = String("some text")
 s.custom_operation() # Calls String.custom_operation()
 print(s.split()) # Calls String.__getattr__("split") and delegates to str.split
 # ['some', 'text']
 
 print("some text" + "more text")
 # ... works
 print(s + "more text")
 # TypeError: unsupported operand type(s) for +: 'String' and 'str'

Copy after login

We want to add some extra functions to the class (like custom_operation above) to define a custom implementation of string. But we don't want to reimplement every string method, such as split, join, capitalize, etc. Here we can use __getattr__ to call these existing string methods.

While this works for normal methods, please note that in the example above, the magic method __add__ (which provides operations such as connections) does not get a delegate. So, if we want them to work as well, we have to reimplement them.

自省(introspection)

最后一个与元编程相关的方法是__getattribute__。它一个看起来非常类似于前面的__getattr__，但是他们有一个细微的区别，__getattr__只在属性查找失败时被调用，而__getattribute__是在尝试属性查找之前被调用。

所以可以使用__getattribute__来控制对属性的访问，或者你可以创建一个装饰器来记录每次访问实例属性的尝试:

 def logger(cls):
original_getattribute = cls.__getattribute__
 
def getattribute(self, name):
print(f"Getting: '{name}'")
return original_getattribute(self, name)
 
cls.__getattribute__ = getattribute
return cls
 
 @logger
 class SomeClass:
def __init__(self, attr):
self.attr = attr
 
def func(self):
...
 
 instance = SomeClass("value")
 instance.attr
 # Getting: 'attr'
 instance.func()
 # Getting: 'func'

Copy after login

装饰器函数logger 首先记录它所装饰的类的原始__getattribute__方法。然后将其替换为自定义方法，该方法在调用原始的__getattribute__方法之前记录了被访问属性的名称。

魔法属性

到目前为止，我们只讨论了魔法方法，但在Python中也有相当多的魔法变量/属性。其中一个是__all__:

 # some_module/__init__.py
 __all__ = ["func", "some_var"]
 
 some_var = "data"
 some_other_var = "more data"
 
 def func():
return "hello"
 
 # -----------
 
 from some_module import *
 
 print(some_var)
 # "data"
 print(func())
 # "hello"
 
 print(some_other_var)
 # Exception, "some_other_var" is not exported by the module

Copy after login

这个属性可用于定义从模块导出哪些变量和函数。我们创建了一个Python模块…/some_module/单独文件(__init__.py)。在这个文件中定义了2个变量和一个函数，只导出其中的2个(func和some_var)。如果我们尝试在其他Python程序中导入some_module的内容，我们只能得到2个内容。

但是要注意，__all__变量只影响上面所示的* import，我们仍然可以使用显式的名称导入函数和变量，比如import some_other_var from some_module。

另一个常见的双下划线变量(模块属性)是__file__。这个变量标识了访问它的文件的路径:

 from pathlib import Path
 
 print(__file__)
 print(Path(__file__).resolve())
 # /home/.../directory/examples.py
 
 # Or the old way:
 import os
 print(os.path.dirname(os.path.abspath(__file__)))
 # /home/.../directory/

Copy after login

这样我们就可以结合__all__和__file__，可以在一个文件夹中加载所有模块:

# Directory structure:
 # .
 # |____some_dir
 # |____module_three.py
 # |____module_two.py
 # |____module_one.py
 
 from pathlib import Path, PurePath
 modules = list(Path(__file__).parent.glob("*.py"))
 print([PurePath(f).stem for f in modules if f.is_file() and not f.name == "__init__.py"])
 # ['module_one', 'module_two', 'module_three']

Copy after login

最后一个我重要的属性是的是__debug__。它可以用于调试，但更具体地说，它可以用于更好地控制断言:

 # example.py
 def func():
if __debug__:
print("debugging logs")
 
# Do stuff...
 
 func()

Copy after login

如果我们使用python example.py正常运行这段代码，我们将看到打印出“调试日志”，但是如果我们使用python -O example.py，优化标志(-O)将把__debug__设置为false并删除调试消息。因此，如果在生产环境中使用-O运行代码，就不必担心调试过程中被遗忘的打印调用，因为它们都不会显示。