Big update to ControlNet: Accurate P drawings can be made by relying only on prompt words, keeping the style of painting unchanged. Netizens: The effect is comparable to a customized large model

Stable Diffusion plug-in, "AI painting detail control master" ControlNet welcomes a major update:

Just use text prompt words to maintain the main characteristics of the image in , you can modify the image details arbitrarily.

For example, give a beautiful girl a different look from hair to clothes, and expression to be more friendly:

Or let the model switch from the sweet girl next door to the cold lady , with the of the body and head facing , Background Change the pattern:

- No matter how the details are modified, the "soul" of the original image is still there.

In addition to this style, it can also handle the anime type just right:

AI design blogger from Twitter @sundyme said:

The effect is better than imagined!

Only one reference picture is needed to complete the above transformation, and some pictures can almost achieve the effect of customized large models.

Ahem, friends in the AI ​​painting circle, cheer up, there is something fun again.

(ps. The first and third renderings are from blogger @Olivio Sarikas, the second one is from Twitter blogger @sundyme)

New on ControlNet: Reserved The original image retouching function

The above update actually refers to a preprocessor called"reference-only".

It does not require any control model and can directly use reference images to guide diffusion.

According to the author, this function is actually similar to the "inpaint" function, but it will not cause the image to collapse.

(Inpaint is a partial redraw function in Stable Diffusion web UI, which can redraw unsatisfactory areas that have been manually masked.)

Some experienced players may know a trick, which is to use inpaint to diffuse images.

For example, you have a 512x512 image of a dog, and then want to generate another 512x512 image using the same dog.

At this time you can connect the 512x512 dog image and the 512x512 blank image into a 1024x512 image, and then use the inpaint function to mask out the blank 512x512 part and diffuse the dog with a similar appearance. image.

In this process, because the images are simply and roughly spliced, and distortion occurs, the effect is generally unsatisfactory.

With "reference-only" it is different:

It can directly link the attention layer of SD (i.e. "Stable Diffusion") to any independent images, so that SD can directly read these images as reference.

In other words, if you want to make modifications while maintaining the style of the original image, you can use the prompt words to operate directly on the original image.

As shown in the official example picture, change a standing puppy into a running action:

You only need to change your Upgrade ControlNet to version 1.1.153 or above, then select "reference-only" as the preprocessor, upload a picture of the dog, enter the prompt word "a dog running on grassland, best quality...", and the SD will I will only use your picture as a reference to make changes.

Netizens: ControlNet’s best feature so far

As soon as the “reference-only” function came out, many netizens started to experience it.

Some people call this the best feature of ControlNet so far:

Upload an anime picture with a character's pose, and then write a prompt that seems completely unrelated to the original picture. Suddenly, the effect you want comes out of the original image. It's really strong, even to the point of changing the rules of the game.

Also known as:

It’s time to pick up all the useless pictures that were discarded before and repair them again.

Of course, there are also people who think it is not so perfect (for example, the beauty’s earrings are wrong in the first rendering at the beginning, and in the second picture The hair inside is also missing), but netizens still said that "the direction is finally right."

The following are the effects of three Twitter bloggers’ attempts, mainly in anime style, let’s enjoy them together:

△From @新宫ラリのAIイラストニュ

△ From @br_d, the original picture is on the left

△From @br_d, the previous one is the original picture

##△ From @uoyuki667, the original picture is on the left Did it touch your heart?

The above is the detailed content of Big update to ControlNet: Accurate P drawings can be made by relying only on prompt words, keeping the style of painting unchanged. Netizens: The effect is comparable to a customized large model. For more information, please follow other related articles on the PHP Chinese website!