Home Technology peripherals AI Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

Mar 02, 2024 am 09:55 AM
industry Redraw

Recently, the "AI Expansion" function has caused a sensation with its sudden enlargement effect. Its funny and interesting auto-fill results have frequently become popular and set off a craze on the Internet. Users actively tried this feature, and its huge 180-degree transformation also made people marvel, and the popularity of the topic continued to rise.

While arousing laughter and enthusiasm, it also means that people are constantly paying attention to whether AI can really help them solve real-world problems and improve user experience. With the rapid development of AIGC technology, AI application scenarios are accelerating to be implemented, which indicates that we will usher in a new productivity revolution.

Recently, Meitu's WHEE and other products have launched AI image expansion and AI image modification functions. With simple prompt input, users can modify images, remove screen elements, and expand the screen at will. With convenience The operation and stunning effects greatly reduce the threshold for using the tool and bring users an efficient and high-quality image creation experience.

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

MiracleVision (Qixiang Intelligence) eliminates the result

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

MiracleVision (Qixiang Intelligence) replaces the effect Before

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

MiracleVision (Qixiang Intelligence) replacement effect After Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

MiracleVision (Qixiang Intelligence) AI image modification effect

Powerful model capabilities, allowing you to edit images as you wish

Meitu AI partial redrawing model builds a complete inpaint & outpaint model framework based on diffusion model (Diffision Model) technology, redrawing internal areas and foregrounds Tasks such as target elimination and external area expansion are unified into one solution, and special optimization designs are made for some specific effect issues.

The MiracleVision model is a Vincentian graph model. Although it can be adapted to the inpaint task by transforming the first convolutional layer and fine-tuning the entire unet, this requires modifying the original weights of unet. , which may lead to a decline in model performance when the amount of training data is insufficient.

Therefore, in order to make full use of MiracleVision's existing generation capabilities, the team does not directly fine-tune MiracleVision's unet model in the partial redraw model, but uses controlnet to add a The input branch of the mask is controlled.

At the same time, in order to save training costs and speed up inference, the compressed controlnet module is used for training to reduce the amount of calculation as much as possible. During the training process, the parameters of the unet model will be fixed, and only the controlnet module will be updated, eventually allowing the entire model to gain the ability to inpaint.

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

Meitu AI partially redraws the model architecture diagram

The outpaint task is the reverse operation of the crop task. The crop task is to crop the original image along the image boundary, retaining only the required part, which is a subtraction operation of the image content; while the outpiant task is to expand outward along the image boundary, using the generation ability of the model to create out of thin air Extracting content that does not originally exist is an addition operation of image content.

Essentially, the outpaint task can also be regarded as a special inpaint task, except that the mask area is located on the periphery of the image.

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

##MiracleVision AI expansion effect

Because the mask area in the outpaint task can only be drawn from the image Guidance information is obtained internally, and other directions are image boundaries, so the generated content is more random and divergent. In order to fill the blank areas on the periphery of the image and ensure the accuracy of the picture expansion, the team relied on scene recognition algorithms to infer the image style and content, and made full use of the correlation of the image content to copy the pixels in the original image by mirroring at the expanded edges. And superimpose random noise to provide a suitable initial prior for the model, thereby ensuring the rationality of the generated content and making the boundary transition smoother.

Freely control the generation and elimination of objects through a variety of training strategies

General diffusion models are better at replacing rather than eliminating when performing inpaint tasks. When a certain target needs to be eliminated, the model can easily draw some new foreground targets in the mask area that do not originally exist, especially when the mask area This phenomenon is especially obvious when the area is relatively large, even if these targets do not appear in the prompt. The reasons are mainly the following three aspects:

1. The prompt of the training set generally only describes what is in the image, but not what is not in the image, so the trained model is allowed to be based on It's easy to tell a prompt to generate a target, but it's hard to stop it from generating a target. Even with the Classifier-Free Guidance strategy, the generation of this target can be suppressed by adding unwanted objects to negative words, but it is never possible to write all possible targets into negative words, so the model will still tend to Generate some unexpected targets;

2. From the distribution of training data, since most of the images in the large-scale image training set are composed of foreground and background, pure background images account for is relatively small, which means that the diffusion model has learned a potential rule during training, that is, there is a high probability of a target foreground in an image (even if it is not mentioned in the prompt), which also causes the model to When performing the inpaint task, it is more likely to generate something in the mask area, so that the output image is closer to the distribution during training;

3. The shape of the mask area to be filled sometimes also contains certain semantic information. , for example, without other guidance, the model will be more inclined to fill a new cat in a mask area with the shape of a cat, causing the elimination task to fail.

In order to make MiracleVision capable of both target generation and target elimination, the team adopted a multi-task training strategy:

1. In the training phase, when the mask area falls on the texture When there are fewer pure background areas, add a specific prompt keyword as a trigger guide word, and during the model inference stage, add this keyword as a forward guide word to the prompt embedding to prompt the model to generate more background areas. .

2. Since pure background images account for a relatively small proportion in the entire training set, in order to increase their contribution to training, in each training batch, a certain proportion of background images are manually sampled and added to the training, so that the background images The proportion in the training samples remains generally stable.

3. In order to reduce the semantic dependence of the model on the mask shape, various masks of different shapes will be randomly generated during the training phase to increase the diversity of mask shapes.

High-precision texture generation, more natural fusion

Since the high-definition texture data in the training set only accounts for a small part of the total training data, when performing the inpaint task, Usually, results with very rich textures are not generated, which results in unnatural fusion and a sense of boundary in scenes with rich original textures.

In order to solve this problem, the team used the self-developed texture detail model as a guide model to assist MiracleVision in improving the generation quality and suppressing over-fitting, so that the generated area is between the generated area and other areas of the original image. can fit together better.

Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want

Original image v.s without added texture details v.sMiracleVision enlarged image effect

Faster, better effect, more efficient interaction!

Diffusion model solutions usually require a multi-step reverse diffusion process during inference, resulting in the processing of a single image taking too long. In order to optimize the user experience while maintaining generation quality, the Meitu Imaging Research Institute (MT Lab) team created a special tuning solution for AI partial redrawing technology, ultimately achieving the best balance between performance and effect.

First of all, a large number of matrix calculations in the pre- and post-processing and inference processes of MiracleVision are transplanted to the GPU for parallel computing as much as possible, thus effectively speeding up the calculation and reducing the load on the CPU. At the same time, during the process of assembling pictures, we fuse the layers as much as possible, use FlashAttention to reduce video memory usage, improve inference performance, and tune the Kernel implementation to maximize the use of GPU computing power for different NVIDIA graphics cards.

In addition, relying on the self-developed model parameter quantification method, MiracleVision is quantized to 8bit without obvious loss of accuracy. Since different GPU graphics cards have different support for 8-bit quantization, we innovatively adopt a mixed-precision strategy to adaptively select the optimal operator under different server resource environments to achieve the optimal solution for overall acceleration.

For user-input images with higher resolutions, it is difficult to perform inference directly at the original resolution due to limitations in server resources and time costs. In this regard, the team first compressed the image resolution to a suitable size, then performed inference based on MiracleVision, and then used a super-resolution algorithm to restore the image to the original resolution, and then performed image fusion with the original image, thereby maintaining both It generates clear images and saves memory usage and execution time during the inference process.

Meitu cooperates in depth with Samsung to create a new mobile image editing experience using AI

On January 25, Samsung Electronics held a new product launch conference for the Galaxy S24 series in China. Meitu has deepened its cooperation with Samsung to create a new AI image editing experience for Samsung's new Galaxy S24 series mobile phone albums. The generative editing independently developed by Meitu Imaging Research Institute (MT Lab) - AI image expansion and AI image modification functions are also It has been officially launched to help open up a new space for mobile image editing and creation.

With the AI ​​image editing function, users can easily move, delete or resize the image by simply pressing and holding the image they want to edit. In addition, when the horizontal line of the picture is not vertical, the AI ​​image expansion function can intelligently fill in the missing areas of the photo and correct the composition of the picture after the user adjusts the angle.

Based on the AI ​​function brought by MiracleVision, Meitu not only helps users easily achieve professional-level editing effects on mobile phones and create more personalized photo works, but will also continue to promote and enhance the entire AI image processing capabilities in the mobile phone industry.

Relying on the powerful technical capabilities of Meitu Imaging Research Institute (MT Lab), MiracleVision has been iterated to version 4.0 in less than half a year. In the future, Meitu will continue to strive to improve user experience in e-commerce, advertising, gaming and other industries, and help practitioners in different scenarios improve workflow efficiency.

The above is the detailed content of Meitu AI partial redrawing technology revealed! Change it however you want! Partial redrawing of beautiful pictures allows you to do whatever you want. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

DeepMind robot plays table tennis, and its forehand and backhand slip into the air, completely defeating human beginners DeepMind robot plays table tennis, and its forehand and backhand slip into the air, completely defeating human beginners Aug 09, 2024 pm 04:01 PM

But maybe he can’t defeat the old man in the park? The Paris Olympic Games are in full swing, and table tennis has attracted much attention. At the same time, robots have also made new breakthroughs in playing table tennis. Just now, DeepMind proposed the first learning robot agent that can reach the level of human amateur players in competitive table tennis. Paper address: https://arxiv.org/pdf/2408.03906 How good is the DeepMind robot at playing table tennis? Probably on par with human amateur players: both forehand and backhand: the opponent uses a variety of playing styles, and the robot can also withstand: receiving serves with different spins: However, the intensity of the game does not seem to be as intense as the old man in the park. For robots, table tennis

The first mechanical claw! Yuanluobao appeared at the 2024 World Robot Conference and released the first chess robot that can enter the home The first mechanical claw! Yuanluobao appeared at the 2024 World Robot Conference and released the first chess robot that can enter the home Aug 21, 2024 pm 07:33 PM

On August 21, the 2024 World Robot Conference was grandly held in Beijing. SenseTime's home robot brand "Yuanluobot SenseRobot" has unveiled its entire family of products, and recently released the Yuanluobot AI chess-playing robot - Chess Professional Edition (hereinafter referred to as "Yuanluobot SenseRobot"), becoming the world's first A chess robot for the home. As the third chess-playing robot product of Yuanluobo, the new Guoxiang robot has undergone a large number of special technical upgrades and innovations in AI and engineering machinery. For the first time, it has realized the ability to pick up three-dimensional chess pieces through mechanical claws on a home robot, and perform human-machine Functions such as chess playing, everyone playing chess, notation review, etc.

Claude has become lazy too! Netizen: Learn to give yourself a holiday Claude has become lazy too! Netizen: Learn to give yourself a holiday Sep 02, 2024 pm 01:56 PM

The start of school is about to begin, and it’s not just the students who are about to start the new semester who should take care of themselves, but also the large AI models. Some time ago, Reddit was filled with netizens complaining that Claude was getting lazy. "Its level has dropped a lot, it often pauses, and even the output becomes very short. In the first week of release, it could translate a full 4-page document at once, but now it can't even output half a page!" https:// www.reddit.com/r/ClaudeAI/comments/1by8rw8/something_just_feels_wrong_with_claude_in_the/ in a post titled "Totally disappointed with Claude", full of

At the World Robot Conference, this domestic robot carrying 'the hope of future elderly care' was surrounded At the World Robot Conference, this domestic robot carrying 'the hope of future elderly care' was surrounded Aug 22, 2024 pm 10:35 PM

At the World Robot Conference being held in Beijing, the display of humanoid robots has become the absolute focus of the scene. At the Stardust Intelligent booth, the AI ​​robot assistant S1 performed three major performances of dulcimer, martial arts, and calligraphy in one exhibition area, capable of both literary and martial arts. , attracted a large number of professional audiences and media. The elegant playing on the elastic strings allows the S1 to demonstrate fine operation and absolute control with speed, strength and precision. CCTV News conducted a special report on the imitation learning and intelligent control behind "Calligraphy". Company founder Lai Jie explained that behind the silky movements, the hardware side pursues the best force control and the most human-like body indicators (speed, load) etc.), but on the AI ​​side, the real movement data of people is collected, allowing the robot to become stronger when it encounters a strong situation and learn to evolve quickly. And agile

Li Feifei's team proposed ReKep to give robots spatial intelligence and integrate GPT-4o Li Feifei's team proposed ReKep to give robots spatial intelligence and integrate GPT-4o Sep 03, 2024 pm 05:18 PM

Deep integration of vision and robot learning. When two robot hands work together smoothly to fold clothes, pour tea, and pack shoes, coupled with the 1X humanoid robot NEO that has been making headlines recently, you may have a feeling: we seem to be entering the age of robots. In fact, these silky movements are the product of advanced robotic technology + exquisite frame design + multi-modal large models. We know that useful robots often require complex and exquisite interactions with the environment, and the environment can be represented as constraints in the spatial and temporal domains. For example, if you want a robot to pour tea, the robot first needs to grasp the handle of the teapot and keep it upright without spilling the tea, then move it smoothly until the mouth of the pot is aligned with the mouth of the cup, and then tilt the teapot at a certain angle. . this

ACL 2024 Awards Announced: One of the Best Papers on Oracle Deciphering by HuaTech, GloVe Time Test Award ACL 2024 Awards Announced: One of the Best Papers on Oracle Deciphering by HuaTech, GloVe Time Test Award Aug 15, 2024 pm 04:37 PM

At this ACL conference, contributors have gained a lot. The six-day ACL2024 is being held in Bangkok, Thailand. ACL is the top international conference in the field of computational linguistics and natural language processing. It is organized by the International Association for Computational Linguistics and is held annually. ACL has always ranked first in academic influence in the field of NLP, and it is also a CCF-A recommended conference. This year's ACL conference is the 62nd and has received more than 400 cutting-edge works in the field of NLP. Yesterday afternoon, the conference announced the best paper and other awards. This time, there are 7 Best Paper Awards (two unpublished), 1 Best Theme Paper Award, and 35 Outstanding Paper Awards. The conference also awarded 3 Resource Paper Awards (ResourceAward) and Social Impact Award (

Hongmeng Smart Travel S9 and full-scenario new product launch conference, a number of blockbuster new products were released together Hongmeng Smart Travel S9 and full-scenario new product launch conference, a number of blockbuster new products were released together Aug 08, 2024 am 07:02 AM

This afternoon, Hongmeng Zhixing officially welcomed new brands and new cars. On August 6, Huawei held the Hongmeng Smart Xingxing S9 and Huawei full-scenario new product launch conference, bringing the panoramic smart flagship sedan Xiangjie S9, the new M7Pro and Huawei novaFlip, MatePad Pro 12.2 inches, the new MatePad Air, Huawei Bisheng With many new all-scenario smart products including the laser printer X1 series, FreeBuds6i, WATCHFIT3 and smart screen S5Pro, from smart travel, smart office to smart wear, Huawei continues to build a full-scenario smart ecosystem to bring consumers a smart experience of the Internet of Everything. Hongmeng Zhixing: In-depth empowerment to promote the upgrading of the smart car industry Huawei joins hands with Chinese automotive industry partners to provide

The first large UI model in China is released! Motiff's large model creates the best assistant for designers and optimizes UI design workflow The first large UI model in China is released! Motiff's large model creates the best assistant for designers and optimizes UI design workflow Aug 19, 2024 pm 04:48 PM

Artificial intelligence is developing faster than you might imagine. Since GPT-4 introduced multimodal technology into the public eye, multimodal large models have entered a stage of rapid development, gradually shifting from pure model research and development to exploration and application in vertical fields, and are deeply integrated with all walks of life. In the field of interface interaction, international technology giants such as Google and Apple have invested in the research and development of large multi-modal UI models, which is regarded as the only way forward for the mobile phone AI revolution. In this context, the first large-scale UI model in China was born. On August 17, at the IXDC2024 International Experience Design Conference, Motiff, a design tool in the AI ​​era, launched its independently developed UI multi-modal model - Motiff Model. This is the world's first UI design tool

See all articles