How to play AI painting, which is very popular this year
1. Foreword
2022 can definitely be said to be the first year of AIGC. Judging from Google search trends, the search volume for AI painting and AI generated art will surge in 2022.
A very important reason for the explosion of AI painting this year is the open source of Stable Diffusion, which is also inseparable from the Diffusion Model in recent years. The rapid development of diffusion models in recent years, combined with OPENAI's already developed text language model GPT-3, makes the generation process from text to images easier.
2. The bottleneck of GAN (Generative Adversarial Network)
From its birth in 2014 to StyleGAN in 2018, GAN has made great progress in the field of image generation. Just like predators and prey in nature compete and evolve together, the principle of GAN is simply to use two neural networks: one as a generator and one as a discriminator. The generator generates different images for the discriminator to judge. Whether the result is qualified or not, the two compete against each other to train the model.
GAN (Generative Adversarial Network) has achieved good results through continuous development, but there are some problems that are always difficult to overcome: lack of diversity in generated results , Mode collapse (the generator stops making progress after finding the best mode), and high training difficulty. These difficulties have made it difficult for AI-generated art to produce practical products.
3. Breakthrough of Diffusion Model
After many years of GAN bottleneck period, scientists came up with a very magical Diffusion Model method to train the model: The original image uses a Markov chain to continuously add noise points to it, and finally becomes a random noise image. Then the training neural network is allowed to reverse this process and gradually restore the random noise image to the original image. In this way, the neural network can It is said to be the ability to generate images from scratch. To generate images from text, the description text is processed and added as noise to the original image. This allows the neural network to generate images from text.
Diffusion Model makes training the model easier. It only requires a large number of pictures, and the quality of the generated images can also be improved. Reaching a very high level, and generating a great diversity of results, this is why the new generation of AI can have unbelievable "imagination".
Of course, technology has been making breakthroughs. The upgraded version of StyleGAN-T launched by NVIDIA at the end of January has made amazing progress. It takes more time to generate a picture than Stable Diffusion under the same computing power. 3 seconds, StyleGAN-T only takes 0.1 seconds. And StyleGAN-T is better than Diffusion Model in low-resolution images, but in the generation of high-resolution images, Diffusion Model still dominates. Since StyleGAN-T is not as widely used as Stable Diffusion, this article will focus on introducing Stable Diffusion.
4. Stable Diffusion
Earlier this year, the AI painting circle experienced the era of Disco Diffusion, DALL-E2, and Midjouney. It was not until Stable Diffusion was open source that it entered a period of time. The dust has settled. As the most powerful AI painting model, Stable Diffusion has caused a carnival in the AI community. Basically, new models and new open source libraries are born every day. Especially after the launch of the WebUI version of Auto1111, using Stable Diffusion has become a very simple matter whether it is deployed in the cloud or locally. With the continuous development of the community, many excellent projects, such as Dreambooth and deforum, have become Stable. A plug-in for the Diffusion WEBUI version has been added, allowing functions such as fine-tuning models and generating animations to be completed in one stop.
5. Introduction to AI painting gameplay and capabilities
The following is an introduction to the gameplay and capabilities currently available using Stable Diffusion
Stable Diffusion capability introduction (the following pictures are output using the SD1.5 model) | ||||||
Introduction |
Input |
Output |
||||
text2img |
Generate pictures through text description, and you can specify the artist style and art type through text description. Here’s an example in the style of artist Greg Rutkowski. |
a beautiful girl with a flowered shirt posing for a picture with her chin resting on her right hand, by Greg Rutkowski |
|
|||
Generate pictures through pictures and text descriptions | a beautiful girl with a flowered shirt posing for a picture with her chin resting on her right hand, by Greg Rutkowski |
|
|
|||
Based on img2img, By setting the mask, only the area within the mask is drawn, which is generally used to modify keywords to fine-tune the screen. | a beautiful girl with a flowered shirt gently smiling posing for a picture with her chin resting on her right hand, by Greg Rutkowski |
|
|
|||
text2img | Currently the most effective two-dimensional animation style model is trained by NAI based on public pictures from the danbooru website as a data set. However, due to copyright issues on danbooru itself, NovelAI has always been controversial, and This model is leaked from commercial services and should be used with caution. a beautiful girl with a flowered shirt posing for a picture with her chin resting on her right hand |
| #NovelAI||||
Use NovelAI’s model for img2img. Yijian AI painting, which is currently very popular in various communities, also uses this ability. But Yiyi mentioned in the disclaimer that their animation model was trained on the data set they collected. | *The text description of the example on the right is based on the image content and AI inference. The artist's style is random##a beautiful girl with a flowered shirt posing for a picture with her chin resting on her right hand |
#AI Painting |
|
Train a model for the subject based on several photos provided by the user. This model can be used to generate any picture containing the subject based on the description. |
This set of pictures uses 20 photos of colleagues to train a 2000 step-out model based on the Stable Diffusion 1.5 model, with several stylized prompt outputs. prompt example (Figure 1): portrait of alicepoizon, highly detailed vfx portrait, unreal engine, greg rutkowski, loish, rhads, caspar david friedrich, makoto shinkai and lois van baarle, ilya kuvshinov, rossdraws, elegent, tom bagshaw, alphonse mucha, global illumination, detailed and intricate environment *alicepoizon is the name given to this character when training this model |
| ##Style model trained based on the same type of style
Use a set of pictures of the same style to train a fine-tuned large model, which can be used to generate pictures with a unified style . |
This set of pictures is generated using the style model fine-tuned through training with Dewu Digital Collection ME.X. | a beautiful girl with a flowered shirt
|
||||
##Leonardo DiCaprio |
||||||
## |
||||||
Scarlett Johansson |
|
6. Introduction to the current main applications
|
##Introduction | ##Sample|||
provided by Jian and other companies provides a more convenient AI painting experience, and you can use many customized large models with different styles. |
|
|||
Two commercial AI painting services. midjouney has its own unique model with a high degree of productization; DallE 2 provides paid API services and has higher-quality generation effects. |
##Lensa, Manjing, etc. provide personal model training services |
|||
|
https://www.php.cn/link/81d7118d88d5570189ace943bd14f142 The current mainstream AI open source community, similar to github, has a large number of Users' own finetuned (fine-tuned) Stable Diffusion-based model can be downloaded and deployed to their own server or local computer. For example, the pix2pix model on the right is a Stable Diffusion model combined with GPT3, which can complete the inpainting function mentioned above through natural language description. |
|
##7. Build a Stable Diffusion WEBUI service by yourself
7.1 Cloud version
Here is built using the cloud computing power provided by AutoDL. You can also use other Platforms such as Google Colab or Baidu Feipiao, etc.
First register an account on AutoDL and rent a cloud host with an A5000/RTX3090 graphics card. https://www.autodl.com/market/list- Create an image on this host. For the image, you can select the packaged algorithm image on www.codewithgpu.com. Here we take the image https://www.codewithgpu.com/i/AUTOMATIC1111/stable-diffusion-webui/Stable-Diffusion-for-NovelAI as an example. Select it and create it.
Run the following command to start Just serve. If you encounter insufficient system disk space, you can also move the stable-diffusion-webui/ folder to the data disk and restart autodl-tmp. If you encounter startup failure, you can configure academic resource acceleration according to the location of your machine.
cd stable-diffusion-webui/ rm -rf outputs && ln -s /root/autodl-tmp outputs python launch.py --disable-safe-unpickle --port=6006 --deepdanbooru
6.2 本地版本
If you have a computer with a good graphics card, you can deploy it locally. Here is an introduction to building the Windows version:
First you need to install Python 3.10.6, and add environment variables to Path- Install git
- Clone Stable Diffusion WEBUI project code to local
- Place the model file in models/Stable-Diffusion directory, related models can be downloaded from https://www.php.cn/link/81d7118d88d5570189ace943bd14f142
- , run webui-user.bat, and access the service through the local computer IP and port 7860.
- 8. Summary
This article introduces some relevant information about AI painting. Interested friends can also deploy the service themselves and try to learn to use DreamBooth or the latest Lora fine-tunes the large model. I believe that in 2023, as the popularity of AIGC continues to increase, our work and life will be greatly changed by AI. The launch of ChatGPT some time ago gave us a huge shock. Just like the ability to search for information when we first entered the Internet, learning to use AI to assist our work will also be a very important ability in the future.
9. Reference materials
From causes to controversies, let’s talk about AI in the first year of AI generative art- https://sspai.com/ post/76277
- https://blog.csdn.net/ qq_45848817/article/details/127808815
- https://theaisummer.com/diffusion-models/
- https://developers.google.com/machine-learning/gan/gan_structure
- https://www.entrogames.com/2022/08/absolute-beginners-guide-to-midjourney- magical-introduction-to-ai-art/
- https://www.technologyreview.com/2022/12/12/1064751/the-viral -ai-avatar-app- lensa-undressed-me-without-my-consent/
- https://www.php.cn/ link/81d7118d88d5570189ace943bd14f142timbrooks/instruct-pix2pix
The above is the detailed content of How to play AI painting, which is very popular this year. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

How to use Go or Rust to call Python scripts to achieve true parallel execution? Recently I've been using Python...

This article describes how to optimize ZooKeeper performance on Debian systems. We will provide advice on hardware, operating system, ZooKeeper configuration and monitoring. 1. Optimize storage media upgrade at the system level: Replacing traditional mechanical hard drives with SSD solid-state drives will significantly improve I/O performance and reduce access latency. Disable swap partitioning: By adjusting kernel parameters, reduce dependence on swap partitions and avoid performance losses caused by frequent memory and disk swaps. Improve file descriptor upper limit: Increase the number of file descriptors allowed to be opened at the same time by the system to avoid resource limitations affecting the processing efficiency of ZooKeeper. 2. ZooKeeper configuration optimization zoo.cfg file configuration

To strengthen the security of Oracle database on the Debian system, it requires many aspects to start. The following steps provide a framework for secure configuration: 1. Oracle database installation and initial configuration system preparation: Ensure that the Debian system has been updated to the latest version, the network configuration is correct, and all required software packages are installed. It is recommended to refer to official documents or reliable third-party resources for installation. Users and Groups: Create a dedicated Oracle user group (such as oinstall, dba, backupdba) and set appropriate permissions for it. 2. Security restrictions set resource restrictions: Edit /etc/security/limits.d/30-oracle.conf

Problems and solutions encountered when using the requests library to crawl web page data. When using the requests library to obtain web page data, you sometimes encounter the...

About Pythonasyncio...

Discussion on the reasons why pipeline files cannot be written when using Scapy crawlers When learning and using Scapy crawlers for persistent data storage, you may encounter pipeline files...

Fastapi ...

Detailed Steps for Restoring Debian Mail Server This article will guide you on how to restore Debian Mail Server. Before you begin, it is important to remember the importance of data backup. Recovery Steps: Backup Data: Be sure to back up all important email data and configuration files before performing any recovery operations. This will ensure that you have a fallback version when problems occur during the recovery process. Check log files: Check mail server log files (such as /var/log/mail.log) for errors or exceptions. Log files often provide valuable clues about the cause of the problem. Stop service: Stop the mail service to prevent further data corruption. Use the following command: su
