How to create a machine learning model using Python-AI-php.cn

Table of Contents

Introduction

Dataset

1. Train the Scikit-learn model

2. Serving the Scikit-learn model

3. Test the model

5.服务多个模型" >5.服务多个模型

6.测试多个模型的准确性

总结

译者介绍

Home

Technology peripherals

How to create a machine learning model using Python

王林

Apr 09, 2023 pm 04:41 PM

python machine learning

Introduction

Have you ever encountered a scenario where when you train a new model, sometimes you don’t want to bother writing Flask Code (Python’s web framework) or containerizing the model and running it in Docker It, just want to use this model immediately through the API?

If you have this need, you definitely want to know about MLServer. It is a Python-based inference server that recently launched the GA (Genetic Algorithms) version, which is a high-performance server designed for production environments.

Use this to ensure that the model built locally is consistent with the model put into the production environment.

This article uses several image models as examples to introduce you how to use MLServer.

Dataset

The data set we want to use is Fashion MNIST. It contains 70,000 grayscale 28x28 pixel clothing images covering 10 different categories (such as tops, dresses, jackets, pants, etc.).

If you want to reproduce the code in this article, be sure to download the files and unzip them into a file called data. Because the files are very large, they are omitted from the GitHub repository.

1. Train the Scikit-learn model

First, we will use the scikit-learn framework to train a support vector machine (SVM) model. We then save the model to a file called Fashion-MNIST.joblib.

Copy after login

Note: The SVM algorithm is not particularly suitable for large data sets due to its quadratic nature. Depending on your hardware, the model in this example will take a few minutes to train.

2. Serving the Scikit-learn model

Through the above steps, we get a model file Fashion-MNIST.joblib. Next let's see how to use MLServer to provide services.

First, install MLServer through the following command: pip install mlserver.

While additional runtime components are optional, to make serving models smoother, we will also install extensions for Scikit-Learn and XGBoost. .

After completing the pip install mlserver-sklearn mlserver-xgboost operation, you need to add the following two configuration files:

(1) settings.json- This contains the configuration of the server itself.

(2)model-settings.json- As the name suggests, this file contains the model configuration to be run.

For the settings.json file, it is enough to define only one parameter:

{ "debug": "true" }

Copy after login

The model-settings.json file requires more information because it needs to know the service model information:

The

{ "name": "fashion-sklearn", "implementation": "mlserver_sklearn.SKLearnModel", "parameters": { "uri": "./Fashion_MNIST.joblib", "version": "v1" } }

Copy after login

name parameter provides a unique identifier for MLServer, which is particularly useful in multi-model serving scenarios (will be discussed later). Defines the implementation to be used to pre-build the server (if any). It is highly coupled with the machine learning framework used to train the model. In our case scikit-learn is used to train the model, so MLServer will be implemented using scikit-learn. Parameters configuration needs to provide the location and version number of the model file.

With the above configuration, we can use the following command to serve our model: mlserver start .

It’s that simple, now the model is running on the local server. At this point, it can already accept requests like HTTP and gRPC (default ports are 8080 and 8081).

3. Test the model

The model has been started and running normally, let's send some requests to test its operation.

We will send a POST request through the following URL:

http://localhost:8080/v2/models//versions//infer

This URL means accessing the previously trained scikit-learn model. Here you only need to replace MODEL_NAME with the model name of fashion-sklearn, and replace VERSION with v1.

The code below shows how to import the test data, make a request to the model server, and then compare the results with the actual labels:

Copy after login

After running the test.py above , you will get the following response from MLServer:

"model_name": "fashion-sklearn", "model_version": "v1", "id": "31c3fa70-2e56-49b1-bcec-294452dbe73c", "parameters": null, "outputs": [ { "name": "predict", "shape": [ 1 ], "datatype": "INT64", "parameters": null, "data": [ 0 ] } ] }

Copy after login

From the response, you can know that MLServer has generated a request ID and automatically added metadata for the model and version of the service request . Once the model is in production, it is important to collect similar metadata. It allows us to log every request for easy auditing and troubleshooting.

You may also notice that MLServer returns an array outputs. Although only one row of data is sent in the request, MLServer processes batch requests and returns them all together. We can even optimize how we handle multiple requests in a production environment through a technique called adaptive batching.

In the above example, you can also find the predicted results of the model. outputs[0].data indicates that the model has labeled the sample as category 0 (the value 0 corresponds to the category t-shirt/top). The same is true for the true label 0 for this sample, so the model got the correct prediction.

4.Training XGBoost model

Through the above example, we understand how to use MLServer to create a single model. Next, let us see how to handle training in different frameworks of multiple models.

Still using the Fashion MNIST data set, but this time the XGBoost model will be trained.

import pandas as pd import xgboost as xgb import time #Load Training Data train = pd.read_csv('../../data/fashion-mnist_train.csv', header=0) y_train = train['label'] X_train = train.drop(['label'], axis=1) dtrain = xgb.DMatrix(X_train.values, label=y_train.values) #Train Model params = { 'max_depth': 5, 'eta': 0.3, 'verbosity': 1, 'objective': 'multi:softmax', 'num_class' : 10 } num_round = 50 start = time.time() bstmodel = xgb.train(params, dtrain, num_round, evals=[(dtrain, 'label')], verbose_eval=10) end = time.time() exec_time = end-start print(f'Execution time: {exec_time} seconds') #Save Model bstmodel.save_model('Fashion_MNIST.json')

Copy after login

上面用于训练 XGBoost 模型的代码与之前用于训练 scikit-learn 模型的代码类似，但是这一次，我们的模型为了兼容 XGBoost 的格式将其保存为Fashion_MNIST.json 文件.

5.服务多个模型

MLServer 的一大优点是支持多模型服务。这意味着不必为部署的每个 ML 模型创建或运行新服务器。使用上面构建的模型，将使用此功能同时为它们提供服务。

当 MLServer 启动时，它将在目录（和任何子目录）中搜索model-settings.json文件。如果您有多个model-settings.json文件，那么它将自动为所有文件提供服务。

注意：您仍然只需要指定根目录中的（服务器配置）文件settings.json

这是目录结构的细分以供参考：

. ├── data │ ├── fashion-mnist_test.csv │ └── fashion-mnist_train.csv ├── models │ ├── sklearn │ │ ├── Fashion_MNIST.joblib │ │ ├── model-settings.json │ │ ├── test.py │ │ └── train.py │ └── xgboost │ ├── Fashion_MNIST.json │ ├── model-settings.json │ ├── test.py │ └── train.py ├── README.md ├── settings.json └── test_models.py

Copy after login

请注意，有两个model-settings.json文件 - 一个用于 scikit-learn 模型，一个用于 XGBoost 模型。

现在可以运行mlserver start .，它将开始处理两个模型的请求。

[mlserver] INFO - Loaded model 'fashion-sklearn' succesfully. [mlserver] INFO - Loaded model 'fashion-xgboost' succesfully.

Copy after login

6.测试多个模型的准确性

现在两个模型都在 MLServer 上运行，我们可以使用测试集中的样本来验证每个模型的准确性。

以下代码向每个模型发送一个批处理请求（包含完整的测试集），然后将预测值与真实标签进行比较。在整个测试集上执行此操作，为我们提供了衡量每个模型准确性的方法，并将最终结果打印出来。

import pandas as pd import requests import json #Import the test data and split the data from the labels test = pd.read_csv('./data/fashion-mnist_test.csv', header=0) y_test = test['label'] X_test = test.drop(['label'],axis=1) #Build the inference request inference_request = { "inputs": [ { "name": "predict", "shape": X_test.shape, "datatype": "FP64", "data": X_test.values.tolist() } ] } #Send the prediction request to the relevant model, compare responses to training labels and calculate accuracy def infer(model_name, version): endpoint = f"http://localhost:8080/v2/models/{model_name}/versions/{version}/infer" response = requests.post(endpoint, json=inference_request) #calculate accuracy correct = 0 for i, prediction in enumerate(json.loads(response.text)['outputs'][0]['data']): if y_test[i] == prediction: correct += 1 accuracy = correct / len(y_test) print(f'Model Accuracy for {model_name}: {accuracy}') infer("fashion-xgboost", "v1") infer("fashion-sklearn", "v1")

Copy after login

结果表明，XGBoost 模型略微优于 SVM scikit-learn 模型：

Model Accuracy for fashion-xgboost: 0.8953 Model Accuracy for fashion-sklearn: 0.864

Copy after login

总结

希望通过上面的描述，您已经了解使用MLServer服务模型大致过程。如果需要了解更多信息，您需要阅读相关文档并查看不同框架的示例。

对于MLFlow用户而言，可以使用 MLServer 直接在 MLFlow 中提供模型，如果您是Kubernetes用户，您应该了解Seldon Core ，它是一个将模型部署到Kubernetes 的开源工具（它在后台使用 MLServer）。

译者介绍

崔皓，51CTO社区编辑，资深架构师，拥有18年的软件开发和架构经验，10年分布式架构经验。曾任惠普技术专家。乐于分享，撰写了很多热门技术文章，阅读量超过60万。《分布式架构原理与实践》作者。

参考链接：https://dzone.com/articles/How-to-Utilize-Python-Machine-Learning-Models

The above is the detailed content of How to create a machine learning model using Python. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress images for free

Clothoff.io

AI clothes remover

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

R.E.P.O. Best Graphic Settings

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Assassin's Creed Shadows: Seashell Riddle Solution

1 weeks ago By DDD

R.E.P.O. How to Fix Audio if You Can't Hear Anyone

3 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Where to find the Crane Control Keycard in Atomfall

1 weeks ago By DDD

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Where is the login entrance for gmail email?

7416

CakePHP Tutorial

1359

What is the format of the account name of steam

win11 activation key permanent

Related knowledge

Is the conversion speed fast when converting XML to PDF on mobile phone? Apr 02, 2025 pm 10:09 PM

The speed of mobile XML to PDF depends on the following factors: the complexity of XML structure. Mobile hardware configuration conversion method (library, algorithm) code quality optimization methods (select efficient libraries, optimize algorithms, cache data, and utilize multi-threading). Overall, there is no absolute answer and it needs to be optimized according to the specific situation.

How to convert XML files to PDF on your phone? Apr 02, 2025 pm 10:12 PM

It is impossible to complete XML to PDF conversion directly on your phone with a single application. It is necessary to use cloud services, which can be achieved through two steps: 1. Convert XML to PDF in the cloud, 2. Access or download the converted PDF file on the mobile phone.

What is the function of C language sum? Apr 03, 2025 pm 02:21 PM

There is no built-in sum function in C language, so it needs to be written by yourself. Sum can be achieved by traversing the array and accumulating elements: Loop version: Sum is calculated using for loop and array length. Pointer version: Use pointers to point to array elements, and efficient summing is achieved through self-increment pointers. Dynamically allocate array version: Dynamically allocate arrays and manage memory yourself, ensuring that allocated memory is freed to prevent memory leaks.

Is there a mobile app that can convert XML into PDF? Apr 02, 2025 pm 09:45 PM

There is no APP that can convert all XML files into PDFs because the XML structure is flexible and diverse. The core of XML to PDF is to convert the data structure into a page layout, which requires parsing XML and generating PDF. Common methods include parsing XML using Python libraries such as ElementTree and generating PDFs using ReportLab library. For complex XML, it may be necessary to use XSLT transformation structures. When optimizing performance, consider using multithreaded or multiprocesses and select the appropriate library.

Recommended XML formatting tool Apr 02, 2025 pm 09:03 PM

XML formatting tools can type code according to rules to improve readability and understanding. When selecting a tool, pay attention to customization capabilities, handling of special circumstances, performance and ease of use. Commonly used tool types include online tools, IDE plug-ins, and command-line tools.

How to convert XML to PDF on your phone? Apr 02, 2025 pm 10:18 PM

It is not easy to convert XML to PDF directly on your phone, but it can be achieved with the help of cloud services. It is recommended to use a lightweight mobile app to upload XML files and receive generated PDFs, and convert them with cloud APIs. Cloud APIs use serverless computing services, and choosing the right platform is crucial. Complexity, error handling, security, and optimization strategies need to be considered when handling XML parsing and PDF generation. The entire process requires the front-end app and the back-end API to work together, and it requires some understanding of a variety of technologies.

How to open xml format Apr 02, 2025 pm 09:00 PM

Use most text editors to open XML files; if you need a more intuitive tree display, you can use an XML editor, such as Oxygen XML Editor or XMLSpy; if you process XML data in a program, you need to use a programming language (such as Python) and XML libraries (such as xml.etree.ElementTree) to parse.

How to convert xml into pictures Apr 03, 2025 am 07:39 AM

XML can be converted to images by using an XSLT converter or image library. XSLT Converter: Use an XSLT processor and stylesheet to convert XML to images. Image Library: Use libraries such as PIL or ImageMagick to create images from XML data, such as drawing shapes and text.

See all articles