A review of deep clustering and related algorithms

Deep clustering is a method that combines deep learning models and clustering algorithms to automatically learn features from data and group the data into categories with similar characteristics. Compared with traditional clustering algorithms, deep clustering can effectively handle high-dimensional, nonlinear and complex data, and has better expressiveness and accuracy. Through deep learning models, deep clustering can learn abstract representations of data to better capture the intrinsic structure and similarities of the data. The advantage of this method is that it can automatically learn the characteristics of the data without manually defining the characteristics, thus reducing the interference of human factors. Deep clustering has wide applications in many fields, such as computer vision, natural language processing, and recommendation systems.

The core idea of ​​deep clustering is to use a deep learning model to reduce the dimensionality of data to a low-dimensional representation and perform clustering in a low-dimensional space. The main steps include data preprocessing, building a deep learning model, training the model to obtain low-dimensional representation, and applying a clustering algorithm for clustering.

1) Establish a deep learning model: Choose a deep learning model suitable for the problem, such as autoencoders, variational autoencoders, generative adversarial networks, etc.

2) Feature extraction: Use deep learning models to extract features from original data and reduce the dimensionality of high-dimensional data to low-dimensional representation.

3) Cluster analysis: Cluster analysis is performed in a low-dimensional space to group the data into categories with similar characteristics.

4) Back propagation: Based on the clustering results, use the back propagation algorithm to update the deep learning model to improve clustering accuracy.

2. Deep clustering algorithm

1) Autoencoder clustering

Auto Encoder clustering is an unsupervised clustering algorithm based on deep learning, which achieves clustering by learning a low-dimensional representation of the data. The basic idea of ​​autoencoder clustering is to map high-dimensional input data to a low-dimensional space through the encoder, and then reconstruct the low-dimensional data back to the original data through the decoder. The steps of the algorithm are as follows:

1. Define the structure of the autoencoder, including an encoder and a decoder, where the encoder maps the input data to a low-dimensional space, and the decoder maps the input data to a low-dimensional space. The dimensional data is reconstructed back to the original data.

2. Use an unsupervised learning algorithm to train the autoencoder, with the goal of minimizing the reconstruction error, that is, the difference between the original data and the reconstructed data.

3. Use the encoder to map the original data to a low-dimensional space, and use a clustering algorithm to cluster the low-dimensional data to obtain the final clustering result.

2) Deep embedding clustering

Deep embedding clustering is an unsupervised clustering algorithm based on deep learning. It learns Embedded representation of data to achieve clustering. The basic idea of ​​deep embedding clustering is to map the original data to a low-dimensional embedding space through multi-layer nonlinear transformation, and use a clustering algorithm to cluster the data in the embedding space. The steps of the algorithm are as follows:

1. Define the structure of the deep embedding network, including multiple nonlinear transformation layers and an embedding layer, where the nonlinear transformation layer maps the original data through learning To a low-dimensional embedding space, the embedding layer is used to cluster the data in the embedding space.

2. Use an unsupervised learning algorithm to train a deep embedding network, with the goal of minimizing the distance between data points in the embedding space while making the distance between different clusters As big as possible.

3. Use the embedding layer to map the original data to a low-dimensional embedding space, and use a clustering algorithm to cluster the data in the embedding space to obtain the final clustering result.

3) Spectral clustering

Spectral clustering is a clustering algorithm based on graph theory, which treats data points as graphs nodes in the graph, the similarity between them is regarded as the edge weight in the graph, and then spectral decomposition is used to divide the graph. The basic idea of ​​spectral clustering is to map data points into a low-dimensional feature space and cluster the data points in the feature space. The steps of this algorithm are as follows:

1. Construct a similarity matrix between data points. Commonly used similarity measures include Euclidean distance, cosine similarity, etc.

2. Construct the Laplacian matrix, including the difference between the degree matrix and the adjacency matrix.

3. Perform spectral decomposition of the Laplacian matrix to obtain eigenvectors and eigenvalues.

4. Select the top k feature vectors and project the data points into a low-dimensional feature space.

5. Use the clustering algorithm to cluster the data points in the feature space to obtain the final clustering result.

4) Hierarchical clustering

Hierarchical clustering is a clustering algorithm based on a tree structure, which converts data points layer by layer are divided into different clusters. The basic idea of ​​hierarchical clustering is to regard each data point as an initial cluster, and then continuously merge the clusters with the highest similarity until a large cluster or a specified number of clusters are finally obtained. The steps of hierarchical clustering are as follows:

1. Calculate the similarity matrix between data points. Commonly used similarity measures include Euclidean distance, cosine similarity, etc.

2. Treat each data point as an initial cluster.

3. Calculate the similarity between each cluster. Commonly used similarity measures include single link, complete link, average link, etc.

4. Continuously merge the clusters with the highest similarity until you finally get a large cluster or a specified number of clusters.

5) Generative Adversarial Network Clustering

Generative Adversarial Network clustering is a clustering algorithm based on Generative Adversarial Network (GAN) , which implements clustering through adversarial learning of generator and discriminator. The basic idea of ​​generative adversarial network clustering is to regard data points as the input of the generator, generate low-dimensional embedding vectors through the generator, and use the discriminator to cluster the embedding vectors. The steps of the algorithm are as follows:

1. Define the structure of the generator and the discriminator, where the generator maps high-dimensional input data to low-dimensional embedding vectors, and the discriminator is used to Embedding vectors are clustered.

2. Use an unsupervised learning algorithm to train the generator and discriminator. The goal is to make the embedding vector generated by the generator as close as possible to the real low-dimensional vector, and to make the discriminator Ability to cluster embedding vectors accurately.

3. Use the generator to map the original data to a low-dimensional embedding space, and use a clustering algorithm to cluster the data in the embedding space to obtain the final clustering result.

6) Deep clustering network

The deep clustering network is an unsupervised clustering algorithm based on deep learning. Train encoders and clusterers to implement clustering. The basic idea of ​​the deep clustering network is to encode the original data into a low-dimensional embedding space through the encoder, and then use the clusterer to cluster the data in the embedding space. The steps of the algorithm are as follows:

1. Define the structure of the deep clustering network, including an encoder and a clusterer, where the encoder maps the original data to a low-dimensional embedding space, Clusterers are used to cluster data in the embedding space.

2. Jointly train deep clustering networks using unsupervised learning algorithms, with the goal of minimizing the distance between data points in the embedding space while minimizing the clustering Clustering error.

3. Use the encoder to map the original data to a low-dimensional embedding space, and use the clusterer to cluster the data in the embedding space to obtain the final clustering result.

7) Deep ensemble clustering

Deep ensemble clustering is a clustering algorithm based on deep learning and ensemble learning. Integrate multiple clustering models to improve clustering accuracy. The basic idea of ​​deep ensemble clustering is to obtain more robust and accurate clustering results by training multiple deep clustering models and then integrating their clustering results. The steps of the algorithm are as follows:

1. Define the structure and hyperparameters of multiple deep clustering models, including encoders, clusterers, optimizers, etc.

2. Use supervised or unsupervised learning algorithms to train multiple deep clustering models with the goal of minimizing the clustering error.

3. Integrate the clustering results of multiple deep clustering models. Commonly used integration methods include voting, weighted average, aggregation, etc.

4. Evaluate and analyze the integrated clustering results, and select the optimal clustering result as the final result.

8) Adaptive clustering network

The adaptive clustering network is a clustering algorithm based on deep learning and adaptive learning , which adapts to changes in data distribution and clustering structure by continuously adjusting the parameters of the clusterer. The basic idea of ​​the adaptive clustering network is to adapt to changes in data distribution by training the clusterer, and at the same time adaptively adjust the parameters of the clusterer according to changes in the clustering structure. The steps of the algorithm are as follows:

1. Define the structure of the adaptive clustering network, including encoder, clusterer, adaptive adjustment module, etc.

2. Use an unsupervised learning algorithm to train the adaptive clustering network. The goal is to minimize the clustering error and continuously adjust the parameters of the clusterer through the adaptive adjustment module.

3. In practical applications, the adaptive clustering network continuously receives new data and adaptively adjusts the parameters of the clusterer according to changes in data distribution and clustering structure, thereby Implement adaptive clustering.

9) Density-based deep clustering

Density-based deep clustering is a density-based clustering algorithm that uses Calculate the density of data points to implement clustering. The basic idea of ​​density-based deep clustering is to regard data points as sample points of density distribution, and implement clustering by calculating the distance and density between sample points. The steps of the algorithm are as follows:

#1. Calculate the density and local density of each data point.

2. Select a density threshold and use data points with density lower than the threshold as noise points.

3. Select a neighborhood radius, regard data points with density higher than the threshold as core points, and regard data points within the neighborhood from the core point as directly density reachable point.

4. Connect the direct density reachable points to form clusters, and divide the remaining density reachable points into corresponding clusters.

5. Exclude noise points from clustering.

The above are some common deep clustering algorithms and their basic ideas and steps. They all have different characteristics and scope of application. You can choose the appropriate algorithm for cluster analysis according to the actual situation.

The above is the detailed content of A review of deep clustering and related algorithms. For more information, please follow other related articles on the PHP Chinese website!