Detailed explanation of svm algorithm

The SVM method maps the sample space into a high-dimensional or even infinite-dimensional feature space (Hilbert space) through a nonlinear mapping p, so that in the original sample Nonlinearly separable problems in space are transformed into linearly separable problems in feature space.                                                                                                                                                                                                                                                                 (Recommended learning: phpstorm)

Simply put, it is dimensionality enhancement and linearization.

Dimension enhancement is to map samples to high-dimensional space. Generally speaking, this will increase the complexity of calculations and even cause "dimensionality disaster", so people rarely pay attention to it.

However, for problems such as classification and regression, it is very likely that a sample set that cannot be processed linearly in a low-dimensional sample space can be linearly divided (or regression) through a linear hyperplane in a high-dimensional feature space. .

SVM (Support Vector Machine), whose Chinese name is support vector machine, is a common discrimination method. In the field of machine learning, it is a supervised learning model, usually used for pattern recognition, classification and regression analysis.

Related concepts

Classifier: A classifier is an algorithm that, given the data of a sample, determines which category the sample belongs to. For example, in stock rise and fall prediction, we believe that the trading volume and closing price of the previous day have an impact on the rise and fall of the next day. Then the classifier predicts the rise and fall of the next day through the trading volume and closing price of the sample. algorithm.

Features: In classification problems, the data input to the classifier are called features. Taking the stock rise and fall prediction problem above as an example, the characteristics are the trading volume and closing price of the previous day.

Linear classifier: A linear classifier is a type of classifier, that is, the basis for determining the classification result is obtained through the linear combination of features, and cannot be based on the nonlinear operation results of the features. Taking the stock rise and fall prediction problem above as an example, the basis for judgment can only be the linear combination of the previous day's trading volume and closing price. The trading volume and closing price cannot be squared or squared.

Origin of linear classifier

In practical applications, we often encounter such a problem: given some data points, they belong to two different classes, Now we need to find a linear classifier to classify this data into two categories.

How to divide it? Split the entire space in half (reminds me of Pangu). Take two-dimensional space as an example. As shown in the figure above, we use a straight line to cut the space. The points on the left side of the straight line belong to category -1 (represented by triangles), and the points on the right side of the straight line belong to category 1 (represented by squares).

If you use mathematical language, it is like this: space is a two-dimensional space composed of X1 and X2. The equation of a straight line is X1 X2 = 1, which is expressed in vector notation as [1,1]^{ T}[X1,X2]-1=0. The point x on the left side of the line means that when x is placed on the left side of the equation, the calculated result is less than 0. In the same way, on the right side, x is put into the left side of the equation, and the calculated result is greater than 0.

The above is the detailed content of Detailed explanation of svm algorithm. For more information, please follow other related articles on the PHP Chinese website!