Essentials for getting started with machine learning: How to implement the perceptron algorithm from scratch in Python-AI-php.cn

Table of Contents

Overview

Perceptron Algorithm

Stochastic Gradient Descent

Dataset

Tutorial

1. Make predictions

2. 训练神经权重

3. 对声纳数据集进行建模

Home

Technology peripherals

Essentials for getting started with machine learning: How to implement the perceptron algorithm from scratch in Python

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB

Apr 13, 2023 pm 12:19 PM

machine learning algorithm sensor

In this tutorial, you will learn how to implement the perceptron algorithm from scratch using Python.

After completing this tutorial, you will learn:

How to train the network weights of the perceptron
How to use the perceptron to make predictions
How to implement the perceptron algorithm for real-world classification problems

Let’s get started.

Overview

This section briefly introduces the perceptron algorithm and Sonar data set, which we will apply later.

Perceptron Algorithm

The perceptron is inspired by the information processing of individual nerve cells called neurons.

Neurons receive input signals through their dendrites and pass electrical signals down into the cell body.

In a similar way, the perceptron accepts input signals from samples of training data, which are weighted and combined in a linear equation called activation.

activation = sum(weight_i * x_i) + bias

Copy after login

The activation is then transformed into an output value or prediction using a transfer function such as a step transfer function.

prediction = 1.0 if activation >= 0.0 else 0.0

Copy after login

In this way, the perceptron is a classification algorithm for problems with two classes (0 and 1) where the two classes can be separated using a linear equation.

It is closely related to linear regression and logistic regression which make predictions in a similar way (e.g. weighted sum of inputs).

The weights of the perceptron algorithm must be estimated from the training data using the stochastic gradient descent algorithm.

Stochastic Gradient Descent

Gradient descent is the process of minimizing a function by following the gradient of a cost function.

This involves knowing the form of the cost as well as the derivative so that from a given point you know the gradient and can move in that direction, say downhill to the minimum.

In machine learning, we can use a technique to evaluate and update the weights for each iteration called stochastic gradient descent to minimize the error of our model on the training data.

The way this optimization algorithm works is that each training instance is presented to the model one at a time. The model makes a prediction for a training instance, calculates the error, and updates the model to reduce the error for the next prediction.

This process can be used to find the set of weights in the model that minimizes the model error on the training data.

For the perceptron algorithm, each iteration, the weight (w) is updated using the following equation:

w = w + learning_rate * (expected - predicted) * x

Copy after login

where w is the weight being optimized and learning_rate is the learning rate that must be configured (e.g. 0.01), (expected - predicted) is the prediction error of the model on the training data attributed to the weights, x is the input value.

Dataset

The dataset we will be using in this tutorial is the Sonar dataset.

This is a data set that describes sonar chirps and returns heuristics for different services. The 60 input variables are return strengths at different angles. This is a binary classification problem that requires a model to differentiate between a metal cylinder and a rock.

It is a well understood data set. All variables are continuous and usually range from 0 to 1. Therefore, we don't have to normalize the input data, which is usually a good place to use the perceptron algorithm. The output variables are the strings "M" (for mine) and "R" (for rock), which we need to convert to integers 1 and 0.

The Zero Rule Algorithm can achieve 53% accuracy by predicting the class with the most observations in the data set (M or Mines).

You can learn more about it in the UCI Machine Learning repository:

https://archive.ics.uci.edu/ml/datasets/Connectionist Bench (Sonar, Mines vs. Rocks)) More information on this dataset. You can also download the dataset for free and place it in your working directory as sonar.all-data.csv.

Tutorial

This tutorial is divided into three parts:

Making predictions
Training network weights
Transform Sonar data Set Modeling

These steps will give you the basics of implementing and applying the perceptron algorithm to your own classification predictive modeling problems.

1. Make predictions

The first step is to develop a function that can make predictions.

This will require the evaluation of candidate weight values in stochastic gradient descent and after the model is finalized, we want to start making predictions on test data or new data.

Here is a function called predict() that predicts the output value of a row given a set of weights.

The first weight is always the bias because it is independent and not responsible for a specific input value.

Copy after login

我们可以设计一个小数据集来测试我们的预测函数。

X1 X2 Y 2.78108362.5505370030 1.4654893722.3621250760 3.3965616884.4002935290 1.388070191.8502203170 3.064072323.0053059730 7.6275312142.7592622351 5.3324412482.0886267751 6.9225967161.771063671 8.675418651-0.2420686551 7.6737564663.5085630111

Copy after login

我们也可以使用之前准备好的权重来为这个数据集做预测。

将所有这些集中起来，我们就可以测试我们的 predict() 函数了，如下：

Copy after login

该函数有两个输入值 X1、X2 和三个权重参数 bias、w1 及 w2。该问题的激活函数有如下的形式：

activation = (w1 * X1) + (w2 * X2) + bias

Copy after login

或者，我们能够手动地选择权重值：

1	`activation = (0.206 * X1) + (-0.234 * X2) + -0.1`

Copy after login

运行这个函数，我们将会得到与期望输出值 (y) 相符合的预测值。

Expected=0, Predicted=0 Expected=0, Predicted=0 Expected=0, Predicted=0 Expected=0, Predicted=0 Expected=0, Predicted=0 Expected=1, Predicted=1 Expected=1, Predicted=1 Expected=1, Predicted=1 Expected=1, Predicted=1 Expected=1, Predicted=1

Copy after login

现在，我们已经准备好使用随机梯度下降法（SGD）来最优化我们的权重值。

2. 训练神经权重

我们可以使用 SGD，来估计出对于训练集的权重值。

SGD 有两个参数：

学习率（Learning Rate）：用来限制每次更新中权重项修正值的大小。
迭代次数（Epochs）：在训练集上训练同时更新权重项的次数。

这两个参数，和训练集一起，都将会是预测函数的输入参数。

在这个函数中，我们需要运行三个循环：

对于每次迭代进行循环；
对于一次迭代中，训练集的每一行进行循环；
对于每一行中，每一个值进行循环。

如你所见，我们在每一次迭代中，对训练集每一行中每一个权值都进行更新。

我们基于现有情况模型预测的「误差」，来对权重值进行更新。误差，是由候选权值计算出来的预测值与（数据集中的）期望值的差。

对每一个输入属性，都有一个权重值，并且这些权_重值都连续更新_。如：

w(t+1)= w(t) + learning_rate * (expected(t) - predicted(t)) * x(t)

Copy after login

偏差项以一种相似的方式更新，不过因为它本身就不与特定的输入值有关，因而在式子中没有输入值的项。

bias(t+1) = bias(t) + learning_rate * (expected(t) - predicted(t))

Copy after login

现在，我们把所有的内容组合到一起。如下所示，在 train_weights() 函数中，它使用 SGD 方法，计算对于给定训练集的权重值。

# Estimate Perceptron weights using stochastic gradient descent def train_weights(train, l_rate, n_epoch): weights = [0.0 for i in range(len(train[0]))] for epoch in range(n_epoch): sum_error = 0.0 for row in train: prediction = predict(row, weights) error = row[-1] - prediction sum_error += error**2 weights[0] = weights[0] + l_rate * error for i in range(len(row)-1): weights[i + 1] = weights[i + 1] + l_rate * error * row[i] print('>epoch=%d, lrate=%.3f, error=%.3f' % (epoch, l_rate, sum_error)) return weights

Copy after login

如你所见，我们也在每次迭代中，记录下了平方误差之和（这始终是一个正值）。因而我们能在外循环的每次迭代中，print 一些有用的信息。

我们也可以在我们上面创建的小规模数据集上，对该函数进行测试。

# Make a prediction with weights def predict(row, weights): activation = weights[0] for i in range(len(row)-1): activation += weights[i + 1] * row[i] return 1.0 if activation >= 0.0 else 0.0 # Estimate Perceptron weights using stochastic gradient descent def train_weights(train, l_rate, n_epoch): weights = [0.0 for i in range(len(train[0]))] for epoch in range(n_epoch): sum_error = 0.0 for row in train: prediction = predict(row, weights) error = row[-1] - prediction sum_error += error**2 weights[0] = weights[0] + l_rate * error for i in range(len(row)-1): weights[i + 1] = weights[i + 1] + l_rate * error * row[i] print('>epoch=%d, lrate=%.3f, error=%.3f' % (epoch, l_rate, sum_error)) return weights # Calculate weights dataset = [[2.7810836,2.550537003,0], [1.465489372,2.362125076,0], [3.396561688,4.400293529,0], [1.38807019,1.850220317,0], [3.06407232,3.005305973,0], [7.627531214,2.759262235,1], [5.332441248,2.088626775,1], [6.922596716,1.77106367,1], [8.675418651,-0.242068655,1], [7.673756466,3.508563011,1]] l_rate = 0.1 n_epoch = 5 weights = train_weights(dataset, l_rate, n_epoch) print(weights)

Copy after login

我们将使用 0.1 的学习率和 5 次迭代，也就是把参数在训练集上更新五次，来训练这个模型。

运行这个案例，它将会在每一次迭代结束后，显示出该次迭代后的平方误差和，并在完成所有迭代后，显示最后的权重集。

>epoch=0, lrate=0.100, error=2.000 >epoch=1, lrate=0.100, error=1.000 >epoch=2, lrate=0.100, error=0.000 >epoch=3, lrate=0.100, error=0.000 >epoch=4, lrate=0.100, error=0.000 [-0.1, 0.20653640140000007, -0.23418117710000003]

Copy after login

你可以看到，这个算法很快学会了「解决」这个问题。

现在我们来试试看，如何在一个实际的数据集上应用这个算法。

3. 对声纳数据集进行建模

在这一节中，我们将使用 SGD 方法，对一个声纳数据集，训练一个感知器模型。

在该例子中，我们假定，在当前的工作目录下，有一名为sonar.all-data.csv 的文件，存储着该数据集。

首先该数据集被载入。数据集中字符串格式的数据被转换为数值型，同时输出值从字符串被转换了 0 或 1 的两个整数值。

通过 load_csv(), str_column_to_float()及str_column_to_int() 三个函数，我们实现了对数据集的读取及预处理。

我们使用「k 倍交叉验证法」（k-fold cross validation）来对学习后的模型在未知数据集上的表现进行评估。也就是说，我们需要建立 k 个模型并估计各模型的平均误差。分类准确性将被用于模型的评估工作中。这些工作在 cross_validation_split(), accuracy_metric() 及 evaluate_algorithm() 函数中被完成。

我们将会使用上面设置的 predict() 和 **train_weights()**函数来训练该模型。同时，我们将会用一个新函数 perceptron() 来将它们组合在一起。如下是完整的例子。

# Perceptron Algorithm on the Sonar Dataset from random import seed from random import randrange from csv import reader # Load a CSV file def load_csv(filename): dataset = list() with open(filename, 'r') as file: csv_reader = reader(file) for row in csv_reader: if not row: continue dataset.append(row) return dataset # Convert string column to float def str_column_to_float(dataset, column): for row in dataset: row[column] = float(row[column].strip()) # Convert string column to integer def str_column_to_int(dataset, column): class_values = [row[column] for row in dataset] unique = set(class_values) lookup = dict() for i, value in enumerate(unique): lookup[value] = i for row in dataset: row[column] = lookup[row[column]] return lookup # Split a dataset into k folds def cross_validation_split(dataset, n_folds): dataset_split = list() dataset_copy = list(dataset) fold_size = int(len(dataset) / n_folds) for i in range(n_folds): fold = list() while len(fold) fold_size: index = randrange(len(dataset_copy)) fold.append(dataset_copy.pop(index)) dataset_split.append(fold) return dataset_split # Calculate accuracy percentage def accuracy_metric(actual, predicted): correct = 0 for i in range(len(actual)): if actual[i] == predicted[i]: correct += 1 return correct / float(len(actual)) * 100.0 # Evaluate an algorithm using a cross validation split def evaluate_algorithm(dataset, algorithm, n_folds, *args): folds = cross_validation_split(dataset, n_folds) scores = list() for fold in folds: train_set = list(folds) train_set.remove(fold) train_set = sum(train_set, []) test_set = list() for row in fold: row_copy = list(row) test_set.append(row_copy) row_copy[-1] = None predicted = algorithm(train_set, test_set, *args) actual = [row[-1] for row in fold] accuracy = accuracy_metric(actual, predicted) scores.append(accuracy) return scores # Make a prediction with weights def predict(row, weights): activation = weights[0] for i in range(len(row)-1): activation += weights[i + 1] * row[i] return 1.0 if activation >= 0.0 else 0.0 # Estimate Perceptron weights using stochastic gradient descent def train_weights(train, l_rate, n_epoch): weights = [0.0 for i in range(len(train[0]))] for epoch in range(n_epoch): for row in train: prediction = predict(row, weights) error = row[-1] - prediction weights[0] = weights[0] + l_rate * error for i in range(len(row)-1): weights[i + 1] = weights[i + 1] + l_rate * error * row[i] return weights # Perceptron Algorithm With Stochastic Gradient Descent def perceptron(train, test, l_rate, n_epoch): predictions = list() weights = train_weights(train, l_rate, n_epoch) for row in test: prediction = predict(row, weights) predictions.append(prediction) return(predictions) # Test the Perceptron algorithm on the sonar dataset seed(1) # load and prepare data filename = 'sonar.all-data.csv' dataset = load_csv(filename) for i in range(len(dataset[0])-1): str_column_to_float(dataset, i) # convert string class to integers str_column_to_int(dataset, len(dataset[0])-1) # evaluate algorithm n_folds = 3 l_rate = 0.01 n_epoch = 500 scores = evaluate_algorithm(dataset, perceptron, n_folds, l_rate, n_epoch) print('Scores: %s' % scores) print('Mean Accuracy: %.3f%%' % (sum(scores)/float(len(scores))))

Copy after login

在交叉验证中，我们取 k 为 3——也就是对每一块数据，都有 208/3 约 70 个记录，会在每次迭代中被用于计算。我们取 0.1 的学习率及 500 的训练迭代次数，来训练模型。

你可以尝试你自己的参数，并且看看你的结果能否战胜我的分数。运行这个例子，将会显示对 3 倍交叉验证中每一块的分数，以及平均的分类正确率。

我们可以看到，这个正确率约为 73%，高于由仅考虑主要类的零规则算法（Zero Rule Algorithm）得到的 50% 的正确率基准值。

Scores: [76.81159420289855, 69.56521739130434, 72.46376811594203] Mean Accuracy: 72.947%

Copy after login

The above is the detailed content of Essentials for getting started with machine learning: How to implement the perceptron algorithm from scratch in Python. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress images for free

Clothoff.io

AI clothes remover

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

How to fix KB5055523 fails to install in Windows 11?

3 weeks ago By DDD

How to fix KB5055518 fails to install in Windows 10?

3 weeks ago By DDD

Roblox: Dead Rails - How To Tame Wolves

4 weeks ago By DDD

Strength Levels for Every Enemy & Monster in R.E.P.O.

4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Roblox: Grow A Garden - Complete Mutation Guide

2 weeks ago By DDD

Hot Tools

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

God-level code editing software (SublimeText3)

Hot Topics

Java Tutorial

1662

CakePHP Tutorial

1418

Laravel Tutorial

1311

PHP Tutorial

1261

C# Tutorial

1234

Related knowledge

This article will take you to understand SHAP: model explanation for machine learning Jun 01, 2024 am 10:58 AM

In the fields of machine learning and data science, model interpretability has always been a focus of researchers and practitioners. With the widespread application of complex models such as deep learning and ensemble methods, understanding the model's decision-making process has become particularly important. Explainable AI|XAI helps build trust and confidence in machine learning models by increasing the transparency of the model. Improving model transparency can be achieved through methods such as the widespread use of multiple complex models, as well as the decision-making processes used to explain the models. These methods include feature importance analysis, model prediction interval estimation, local interpretability algorithms, etc. Feature importance analysis can explain the decision-making process of a model by evaluating the degree of influence of the model on the input features. Model prediction interval estimate

Implementing Machine Learning Algorithms in C++: Common Challenges and Solutions Jun 03, 2024 pm 01:25 PM

Common challenges faced by machine learning algorithms in C++ include memory management, multi-threading, performance optimization, and maintainability. Solutions include using smart pointers, modern threading libraries, SIMD instructions and third-party libraries, as well as following coding style guidelines and using automation tools. Practical cases show how to use the Eigen library to implement linear regression algorithms, effectively manage memory and use high-performance matrix operations.

Five schools of machine learning you don't know about Jun 05, 2024 pm 08:51 PM

Machine learning is an important branch of artificial intelligence that gives computers the ability to learn from data and improve their capabilities without being explicitly programmed. Machine learning has a wide range of applications in various fields, from image recognition and natural language processing to recommendation systems and fraud detection, and it is changing the way we live. There are many different methods and theories in the field of machine learning, among which the five most influential methods are called the "Five Schools of Machine Learning". The five major schools are the symbolic school, the connectionist school, the evolutionary school, the Bayesian school and the analogy school. 1. Symbolism, also known as symbolism, emphasizes the use of symbols for logical reasoning and expression of knowledge. This school of thought believes that learning is a process of reverse deduction, through existing

Is Flash Attention stable? Meta and Harvard found that their model weight deviations fluctuated by orders of magnitude May 30, 2024 pm 01:24 PM

MetaFAIR teamed up with Harvard to provide a new research framework for optimizing the data bias generated when large-scale machine learning is performed. It is known that the training of large language models often takes months and uses hundreds or even thousands of GPUs. Taking the LLaMA270B model as an example, its training requires a total of 1,720,320 GPU hours. Training large models presents unique systemic challenges due to the scale and complexity of these workloads. Recently, many institutions have reported instability in the training process when training SOTA generative AI models. They usually appear in the form of loss spikes. For example, Google's PaLM model experienced up to 20 loss spikes during the training process. Numerical bias is the root cause of this training inaccuracy,

Explainable AI: Explaining complex AI/ML models Jun 03, 2024 pm 10:08 PM

Translator | Reviewed by Li Rui | Chonglou Artificial intelligence (AI) and machine learning (ML) models are becoming increasingly complex today, and the output produced by these models is a black box – unable to be explained to stakeholders. Explainable AI (XAI) aims to solve this problem by enabling stakeholders to understand how these models work, ensuring they understand how these models actually make decisions, and ensuring transparency in AI systems, Trust and accountability to address this issue. This article explores various explainable artificial intelligence (XAI) techniques to illustrate their underlying principles. Several reasons why explainable AI is crucial Trust and transparency: For AI systems to be widely accepted and trusted, users need to understand how decisions are made

Improved detection algorithm: for target detection in high-resolution optical remote sensing images Jun 06, 2024 pm 12:33 PM

01 Outlook Summary Currently, it is difficult to achieve an appropriate balance between detection efficiency and detection results. We have developed an enhanced YOLOv5 algorithm for target detection in high-resolution optical remote sensing images, using multi-layer feature pyramids, multi-detection head strategies and hybrid attention modules to improve the effect of the target detection network in optical remote sensing images. According to the SIMD data set, the mAP of the new algorithm is 2.2% better than YOLOv5 and 8.48% better than YOLOX, achieving a better balance between detection results and speed. 02 Background & Motivation With the rapid development of remote sensing technology, high-resolution optical remote sensing images have been used to describe many objects on the earth’s surface, including aircraft, cars, buildings, etc. Object detection in the interpretation of remote sensing images

Machine Learning in C++: A Guide to Implementing Common Machine Learning Algorithms in C++ Jun 03, 2024 pm 07:33 PM

In C++, the implementation of machine learning algorithms includes: Linear regression: used to predict continuous variables. The steps include loading data, calculating weights and biases, updating parameters and prediction. Logistic regression: used to predict discrete variables. The process is similar to linear regression, but uses the sigmoid function for prediction. Support Vector Machine: A powerful classification and regression algorithm that involves computing support vectors and predicting labels.

Application of algorithms in the construction of 58 portrait platform May 09, 2024 am 09:01 AM

1. Background of the Construction of 58 Portraits Platform First of all, I would like to share with you the background of the construction of the 58 Portrait Platform. 1. The traditional thinking of the traditional profiling platform is no longer enough. Building a user profiling platform relies on data warehouse modeling capabilities to integrate data from multiple business lines to build accurate user portraits; it also requires data mining to understand user behavior, interests and needs, and provide algorithms. side capabilities; finally, it also needs to have data platform capabilities to efficiently store, query and share user profile data and provide profile services. The main difference between a self-built business profiling platform and a middle-office profiling platform is that the self-built profiling platform serves a single business line and can be customized on demand; the mid-office platform serves multiple business lines, has complex modeling, and provides more general capabilities. 2.58 User portraits of the background of Zhongtai portrait construction

See all articles