Detailed explanation of K-means++ algorithm in Python

PHPz
Release: 2023-06-10 20:25:44
Original
1838 people have browsed it

K-means algorithm is a common unsupervised learning algorithm used to cluster data into different categories. The K-means algorithm is an improved version of the K-means algorithm, aiming to improve the efficiency and accuracy of initial cluster center selection. This article will introduce in detail the principle, code implementation and application of the K-means algorithm in Python.

  1. Overview of K-means algorithm

K-means algorithm is an iterative algorithm. The process of each iteration is: first randomly select K initial clustering centers, Then each data point is assigned to the category of the initial cluster center closest to it, and then the centers of all clusters are recalculated and the cluster centers are updated. Repeat the above process until the convergence conditions are met.

K-means algorithm process:

  1. Randomly select K data points from the data as the initial clustering center.
  2. Assign data points to the cluster with the nearest cluster center.
  3. Recalculate the center of each cluster.
  4. Repeat 2-3 until the convergence conditions are met (the cluster center no longer changes, the maximum number of iterations is reached, etc.).
  5. K-means algorithm steps

The K-means algorithm is an improved version of the K-means algorithm, which is mainly optimized in the selection of the initial clustering center. The initial cluster center selection steps of the K-means algorithm are as follows:

  1. Randomly select a data point as the first cluster center.
  2. For each data point, calculate its distance D(x) from the nearest cluster center.
  3. Randomly select a data point as the next clustering center. Make sure that the greater the distance between this point and the existing clustering center, the greater the probability of being selected:
    a. Calculate each data The nearest distance D(x)^2 between the point and the existing cluster center.
    b. Calculate the sum Sum(D(x)^2) of all D(x)^2.
    c. Weight each data point according to the proportion of its nearest distance to the existing cluster center, and the probability is D(x)^2/Sum(D(x)^2).
    d. From sampling according to the above probability, select a data point as the next cluster center.
  4. Repeat step 3 until K cluster centers are selected.
  5. Python implements the K-means algorithm

Next, we will implement the K-means algorithm through Python.

First, import the necessary libraries:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
Copy after login

Next, we generate a set of data for clustering:

n_samples = 1500
random_state = 170
X, y = make_blobs(n_samples=n_samples, random_state=random_state)
Copy after login

Then, we train through the KMeans module of sklearn K-means model:

kmeans = KMeans(init="k-means++", n_clusters=3, n_init=10)
kmeans.fit(X)
Copy after login

Finally, we visualize the clustering results:

plt.figure(figsize=(12, 12))
h = 0.02
x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = kmeans.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.imshow(Z, interpolation="nearest",
          extent=(xx.min(), xx.max(), yy.min(), yy.max()),
          cmap=plt.cm.Pastel1, aspect="auto", origin="lower")
plt.scatter(X[:, 0], X[:, 1], s=30, c=kmeans.labels_, cmap=plt.cm.Paired)
plt.scatter(kmeans.cluster_centers_[:, 0], kmeans.cluster_centers_[:, 1],
            marker="^", s=100, linewidths=3,
            color='black', zorder=10)
plt.title("K-means++ clustering")
plt.xlim(x_min, x_max)
plt.ylim(y_min, y_max)
plt.show()
Copy after login
  1. K-means algorithm application scenario

K-means algorithm Suitable for data clustering problems without label information. Compared with the K-means algorithm, in order to ensure the rationality and uniqueness of the initial clustering center, the K-means algorithm is more suitable for situations where there is a lot of data or the data distribution is relatively scattered.

K-means algorithm can be used in data mining, image processing, natural language processing and other fields. Clustering algorithms can be used to find samples with higher similarity, which is also very useful for the visualization of big data.

In short, the K-means algorithm has good application prospects in data mining, cluster analysis, image recognition, natural language processing and other fields.

The above is the detailed content of Detailed explanation of K-means++ algorithm in Python. For more information, please follow other related articles on the PHP Chinese website!

Related labels:
source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template