Learn to automatically import missing libraries in python

python tutorial column introduces how to automatically import missing libraries.

Import failure problems are usually divided into two types: one is to import a module written by yourself (that is, a file with the suffix .py), The other is to import third-party libraries. This article mainly discusses the second situation. If we have the opportunity in the future, we will discuss other related topics in detail.

To solve the problem of failing to import the Python library, the key is actually to install the missing library in the running environment (note whether it is a virtual environment), or to use an appropriate alternative. This problem is divided into three situations:

1. Missing libraries in a single module

When writing code, if we need to use a third-party library (such as requests), but you are not sure whether it is installed in the actual running environment, then you can do this:

try:    import requestsexcept ImportError:    import os
    os.system('pip install requests')    import requests复制代码

The effect of writing this is that if the requests library cannot be found, install it first, Import again.

In some open source projects, we may also see the following writing (taking json as an example):

ry:    import simplejson as jsonexcept ImportError:    import json复制代码

The effect of writing this way is that priority is given to importing Third-party library simplejson, if not found, use the built-in standard library json.

The advantage of this way of writing is that there is no need to import additional libraries, but it has a disadvantage, that is, it needs to ensure that the two libraries are compatible in use. If an alternative library cannot be found in the standard library , then it is not feasible.

If you really can't find a compatible standard library, you can also write a module yourself (such as my_json.py) to achieve what you want, and then except statement to import it.

ry:    import simplejson as jsonexcept ImportError:    import my_json as json复制代码

Code words are not easy to talk nonsense. Two sentences: If you need learning materials or have technical questions, just "Click"

2. Missing libraries in the entire project

The above idea is for projects under development, but it has several shortcomings:

1. pip install for every third-party library that may be missing in the code , is not advisable;

2. What should I do if a third-party library cannot be replaced by the standard library or my own handwritten library?

3. What should I do if these modifications are not allowed for a completed project?

So the question here is: There is a project that I want to deploy to a new machine. It involves many third-party libraries, but none of them are pre-installed on the machine. What should I do?

For a compliant project, according to convention, it usually contains a "requirements.txt" file, which records all the dependent libraries of the project and their required version numbers. This is generated using the command pip freeze > requirements.txt before the project is released.

Use the command pip install -r requirements.txt (execute it in the directory where the file is located, or write the path of the entire file in the command) to automatically install all dependent libraries. superior.

But what if the project is non-compliant, or for some other unlucky reason we don’t have such a document?

A stupid way is to run the project, wait for it to error, and if an import library fails, install it manually, then run the project again, and if the import library fails, install it, and so on... …

3. Automatically import any missing libraries

Is there a better way to automatically import missing libraries?

Is there any way to automatically import the required libraries without modifying the original code and without the need for the "requirements.txt" file?

Of course! Let’s take a look at the effect first:

Let’s take tornado as an example. As you can see from the first step, we have not installed tornado##. #, after the second step, when importing tornado again, the program will automatically download and install tornado for us, so no errors will be reported.

autoinstall is our handwritten module, the code is as follows:

# 以下代码在 python 3.6.1 版本验证通过import sysimport osfrom importlib import import_moduleclass AutoInstall():
    _loaded = set()    @classmethod
    def find_spec(cls, name, path, target=None):
            if path is None and name not in cls._loaded:
                cls._loaded.add(name)
                print("Installing", name)                try:
                    result = os.system('pip install {}'.format(name))                    if result == 0:                        return import_module(name)                except Exception as e:
                    print("Failed", e)            return Nonesys.meta_path.append(AutoInstall)复制代码

sys.meta_path is used in this code, let’s print it first, See what it is?

1. ##The

   import mechanism of Python 3 during the search process, the approximate sequence is as follows:

Look in
2. sys.modules

   , which caches all imported modules

In
3. sys.meta_path

   Search in, it supports custom loaders

Search in
4. sys.path

   , it records the directory names where some libraries are located

If not found, throw
5. ImportError

   Exception

It should be noted that sys.meta_path is different in different Python versions. For example, it is different between Python 2 and There is a big difference in Python 3; in newer Python 3 versions (3.4), custom loaders need to implement the find_spec method, while earlier The version used is find_module.

The above code is a custom class library loader AutoInstall, which can automatically import third-party libraries. It should be noted that this method will "hijack" all newly imported libraries and destroy the original import method, so some strange problems may also occur, so please pay attention.

sys.meta_path is an application of Python probe. Probes, or import hooks, are Python mechanisms that get almost no attention, but they can do a lot of things, such as loading libraries on the network and testing modules when importing them. Modify, automatically install missing libraries, upload audit information, delay loading, etc.

Limited by space limitations, we will not elaborate further. Finally, a summary:

1. You can use try...except to implement simple third-party library import or replacement

2. has been When you know all the missing dependent libraries (such as requirements.txt), you can manually install

3. Use sys.meta_path to automatically import any The missing library

Related free learning recommendations:python tutorial(Video)

The above is the detailed content of Learn to automatically import missing libraries in python. For more information, please follow other related articles on the PHP Chinese website!