AI produces pictures faster, more beautifully, and understands your thoughts better. What technical secrets has the high-beauty Vincent picture model cultivated?

With the launch of large models and pressing the accelerator button, Vincentian diagrams are undoubtedly one of the hottest application directions.

Since the birth of Stable Diffusion, there have been an endless stream of large models of Wen Shengtu at home and abroad, and it felt like "fighting between gods" for a while. In just a few months, the title of "The Strongest AI Artist" has changed hands several times. Every technological iteration continues to push the upper limit of AI image generation quality and speed.

So now, we can get any picture we want by entering a few words. Whether it’s a professional-level commercial poster or a hyper-realistic photo, the fidelity of AI mapping has amazed us. Even AI won the 2023 Sony World Photography Award. Before the announcement of the grand prize, this "photo" had been exhibited at Somerset House in London - if the author did not disclose it publicly, no one might find out that the photo was actually created by AI. E Eldagse and his AI generation work "Electrician"

How to make the pictures drawn by AI is more beautiful, which is inseparable from AI technicians to persevere.

The sixth issue of "AIGC Experience School" invited Doubao Vincent Chart technical expert Li Liang and NVIDIA solution architect Zhao Yijia to provide us with an in-depth analysis of the reasons why the Vincent Chart model produces pictures that are more beautiful, faster, and better understands the user's intentions. Technical links.

The live broadcast began. Li Liang first dissected in detail the recent "top-tier" domestic large model - the technical upgrade of the Vincent diagram model of the ByteDance Doubao large model.

Li Liang said that the problems that the Doubao team wants to solve mainly include three aspects: first, how to achieve stronger image and text matching to meet the user's idea design; second, how to generate more beautiful images to provide more ultimate The user experience; the third is how to produce pictures more quickly to meet ultra-large-scale service calls. In terms of image and text matching, the Doubao team started with data, refined and filtered the massive image and text data, and finally stored hundreds of billions of high-quality images in the database. In addition, the team also specially trained a multi-modal large language model for the recapiton task. This model will more comprehensively and objectively describe the physical relationships of images in pictures.

After having high-quality and high-detailed image and text data, if you want to better leverage the strength of the model, you need to improve the ability of the text understanding module. The team uses a native bilingual large language model as a text encoder, which significantly improves the model's ability to understand Chinese. Therefore, in the face of national elements such as "Tang Dynasty" and "Lantern Festival", the Doubao and Vincent diagram models also show a more profound understanding.

For the Diffsuion model architecture, the Doubao team also injected unique secrets. They used UNet to effectively scale. By increasing the number of parameters, the Doubao·Vensen graph model further improved the understanding of image-text pairs and high-fidelity generation. ability.

Aiming at the most obvious aesthetic style that users intuitively feel, the Doubao team has introduced professional aesthetic guidance and always pays attention to the aesthetic preferences of users and the public. At the same time, the team also worked hard on data and model architecture. Many times, the comparison between the images the user gets and the demo display is like a "buyer show" and a "seller show". In fact, the prompt given is not detailed and clear enough for the model, and the Doubao Vincent diagram model introduces a "Rephraser", while following the user's original intention, adds more detailed descriptions to the prompt words, so all users will experience a more perfect generation effect.

In order to make the model produce pictures faster and consume less cost per picture, the Doubao team also gave new problem-solving ideas in the distillation method of the model. One representative result is Hyber- SD, a novel diffusion model distillation framework that maintains near-lossless performance while compressing the number of denoising steps.

接下来，英伟达解决方案架构师赵一嘉从底层技术出发，讲解了文生图最主流的基于Unet的SD和DIT两种模型架构及其相应的特性，并介绍了英伟达的Tensorrt, Tensorrt-LLM, Triton, Nemo Megatron 等工具如何为部署模型提供支持，助力大模型更加高效地推理。

赵一嘉首先分享了 Stable Diffusion 背后模型的原理详解，细致地阐述了 Clip、VAE 和 Unet 等关键组件的工作原理。随着 Sora 爆火，也带火了背后的 DiT（扩散 Transformer）架构。赵一嘉进一步从模型结构、特性和算力消耗三方面，从模型结构、特性和资源消耗三个方面，对 SD 和 DiT 的优势进行了全面的比较。

使用 Stable diffusion 生成图像时，往往会感觉提示词内容在生成结果中都得到了呈现，但图不是自己想要的，这是因为基于文字出图的 Stable diffusion 并不擅长控制图像的细节，例如构图、动作、面部特征、空间关系等。因此，基于Stable diffusion 的工作原理，研究人员们设计了许多控制模块，弥补 Stable diffusion 的短板。赵一嘉补充了其中具有代表性的 IP-adapter 和 ControlNet。

想要加快吃算力的文生图模型的推理速度，英伟达的技术支持发挥了关键作用。赵一嘉介绍了 Nvidia TensorRT 和 TensorRT-LLM 工具，这些工具通过高性能卷积、高效调度和分布式部署等技术，优化了图文生成模型的推理过程。同时，英伟达的 Ada、Hopper 以及即将推出的 BlackWell 硬件架构，都已支持 FP8 训练和推理，将为模型训练带来更加丝滑的体验。

经历了六场精彩的直播，由火山引擎、NVIDIA 联手本站和 CMO CLUB 共同推出的《AIGC体验派》迎来了圆满收官。通过这六期节目，相信大家对 AIGC 如何从「有趣」变为「有用」有了更深的理解。我们也期待着《AIGC 体验派》不止停留在节目的讨论中，并更能在实际中加速营销领域智能化升级的进程。

《AIGC 体验派》全六期回顾地址：https://vtizr.xetlk.com/s/7CjTy

The above is the detailed content of AI produces pictures faster, more beautifully, and understands your thoughts better. What technical secrets has the high-beauty Vincent picture model cultivated?. For more information, please follow other related articles on the PHP Chinese website!