Table of Contents
Complete "Painting Skin" in 30 seconds
It is said that AI will replace humans. In fact, it is not that easy to replace just in the field of Text-to-3D.
Home Technology peripherals AI Get a virtual 3D wife in 30 seconds with a single card! Text to 3D generates a high-precision digital human with clear pore details, seamlessly connecting with Maya, Unity and other production tools

Get a virtual 3D wife in 30 seconds with a single card! Text to 3D generates a high-precision digital human with clear pore details, seamlessly connecting with Maya, Unity and other production tools

May 23, 2023 pm 02:34 PM
chatgpt 3d

ChatGPT has injected a dose of chicken blood into the AI ​​industry. Everything that was once unimaginable has become basic practice today.

is continuing to advance Text-to-3D, which is regarded as following Diffusion(image) and GPT(text). The next frontier hot spot in the field of AIGC has received unprecedented attention.

No, a product called ChatAvatar has been put into low-key public beta. It quickly attracted more than 700,000 views and attention, and was listed in the Hot (Spaces of the week) .

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

△ChatAvatar will also support Image to 3D technology that generates 3D stylized characters from AI-generated single-perspective/multi-perspective original paintings, which has been widely received Pay attention to

The 3D model generated by the current beta version can be directly downloaded to the local together with the PBR material. Not only does it work well, but more importantly it is free to play. Some netizens exclaimed:

It’s so cool, I feel like I can easily generate my own digital twin.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

#This has attracted many netizens to try it out and contribute their ideas. Some people combined this product with ControlNet and found that the effect was so delicate and realistic that it was unexpected.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

This Text-to-3D tool with almost zero threshold to use is called ChatAvatar and was created by the domestic AI startup Yingmo Technology Team.

It is understood that this is the world’s first Production-Ready Text to 3D product. Through simple text, such as the name of a star or the appearance of a desired character, it can generate film and television-level images. 3D hyper-realistic digital human assets.

The efficiency is also very high. It only takes 30 seconds on average to make a face that looks real - even your own.

In the future, the generation field will also be expanded to other three-dimensional assets.

And the model has regular topology, PBR material with 4k resolution, and binding. It can be directly connected to the production pipeline of production engines such as Unity, Unreal Engine, and Maya.

So, what kind of 3D generation tool is ChatAvatar? What technology is used behind it?

Complete "Painting Skin" in 30 seconds

Experience the gameplay of ChatAvatar personally and find that it can be said to be a truly zero threshold.

Specifically, you only need to describe your needs to ChatBot in vernacular on the official website in the form of a conversation, and a 3D face can be generated on demand and covered with a sticker The real "human skin" of the model.

During the entire conversation process, ChatBot will guide according to the user's needs to understand the user's thoughts on the required model in as much detail as possible.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

During the experience, we described to ChatBot such a 3D image we want to generate:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

Click left Click the Generate button on the side, and in less than 10 seconds on average, the initial prototypes of 9 different 3D faces generated according to the description will appear on the screen.

After selecting one of them at will, the model and material will continue to be optimized based on the selection. Finally, the model rendering result after covering the skin will appear, and the rendering effect under different light and shadow will be displayed - these renderings are completed in real time in the browser :

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

## Use the mouse to drag, you can also rotate the head, and zoom in to see more detailed local effects, pores and acne are clearly visible:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

It is worth mentioning that if the user is a prompt engineering expert, he can also complete the generation by directly entering prompt in the left box.

Finally, with one click download, you can get a 3D digital head asset that can be directly connected to the production engine and driven:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

Although the beta version The hairstyle function has not yet been launched, but overall, the final generated 3D digital human assets and description content have a high degree of matching.

The official website also displays many assets generated by ChatAvatar users, with different races, skin colors, different ages, joy, anger, sorrow, beauty, ugly, fat and thin, and all kinds of looks.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

Let’s summarize the highlights of the ChatAvatar product for generating 3D digital human assets:

First of all, it is easy to use; secondly,The generation span is large, and the facial features can be changed, and masks, tattoos, etc. that fit the face can also be generated, such as this:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

##According to the official According to the promotional video, ChatAvatar can even further generate characters beyond human scope, such as characters in film and television works such as Avatar:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具##The most important thing is, ChatAvatar

Solve the compatibility issues between 3D models and traditional rendering software

. This means that the 3D assets generated by ChatAvatar can be directly integrated into the game and film and television production processes.

Of course, before being officially connected to the industrial process, ChatAvatar has already attracted thousands of artists and professional art personnel to participate in the first round of public beta, and related topics on Twitter have received nearly one million views and attention.

Any tweet can have more than 50k views.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具 It’s not for nothing that I have accumulated a lot of “tap water”. Look at the 3D face of Einstein. Who wouldn’t say that it really looks like him?

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具If combined with ControlNet, the generated effect is no less than that of a SLR photo taken directly:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具There are already many users After the experience, I began to imagine using this Text-to-3D tool on a large scale in industrial applications such as games, film and television.

It is understood that user feedback will become an important basis for the ChatAvatar team to quickly iterate and update, forming a data flywheel to provide more complete and demand-based functions in a timely manner.

In fact, for previous designers or companies in the 3D industry, most AI text-to-3D applications are not ineffective, but there are still many difficulties in actually implementing them into the industrial design process.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具 What are the technical reasons behind ChatAvatar being able to make such a big splash this time?

What is the difficulty in generating 3D assets that meet industry requirements?

It is said that AI will replace humans. In fact, it is not that easy to replace just in the field of Text-to-3D.

The biggest difficulty is to make the things generated by AI meet the industry's requirements for 3D assets from the

standard

. How do you understand the

Industry Standard

here? From the perspective of professional 3D art design, there are at least three aspects-Quality, controllability and generation speed.

The first is quality. Especially for the film, television and game industries that emphasize visual effects, in order to generate 3D assets that meet pipeline requirements, "industry unspoken rules" such as topological regularity and texture mapping accuracy are the first steps that must be taken for AI products. Hom.

Take the regularity of the topological structure as an example. This essentially refers to the reasonableness of the 3D asset wiring.

For 3D assets, the regularity of the topology often directly affects the animation effect, modification processing efficiency and texture drawing speed of the object:

According to the introduction of 3D art design in the industry, manual retopology The time cost is often higher than the production of the 3D model itself, even in multiples. This means that no matter how cool the 3D assets generated by the AI ​​model are, if the generated topological regularity does not meet the requirements, the cost cannot be fundamentally reduced. Not to mention texture accuracy.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

△Yingmo Technology’s ChatAvatar project has significantly improved compared to previous work in terms of generation quality, speed and standard compatibility

Take PBR textures, which are currently commonly required by the game and film and television industries, as an example. They include a series of textures such as reflectivity maps and normal maps, which are equivalent to the "layers" of 2D image PSD files and are essential for 3D asset pipeline production. One of the few conditions.

However, the current 3D assets generated by AI are often a "whole", and it is rare to be able to independently generate PBR textures that meet the industrial environment as required.

The second is controllability. For generative AI, how to make the generated content more "controllable" is another major requirement of the CG industry for this technology.

Take the well-known 2D industry as an example. Before the emergence of ControlNet, the 2D AIGC industry had been in a state of "semi-dark progress".

In other words, AI can generate images of objects of specified categories, but cannot generate objects of specified postures. The generation effect depends entirely on prompt engineering and "metaphysics."

After the emergence of ControlNet, the controllability of 2D AI image generation has been improved by leaps and bounds. However, for 3D AI, in order to generate assets with corresponding effects, it still relies heavily on professional Prompt works.

Finally is the generation speed. Compared with 3D art design, the advantage of AI generation is speed. However, if the speed and effect of AI rendering cannot match that of manual rendering, then this technology will still not be able to bring benefits to the industry.

Take NeRF, which is currently very popular in AI technology, as an example. Its industrialization is faced with compatibility problems of speed and quality.

When the generation quality is high, 3D generation based on NeRF often takes a long time; however, if speed is pursued, even 3D assets generated by NeRF will not be put into industrial use at all.

But even if this problem is solved, how to make NeRF compatible with mainstream engines in the traditional CG industry without losing accuracy is still a huge problem.

It is not difficult to find from the above industrial standardization process that there are two major bottlenecks in the implementation of most AI text to 3D applications:

One is that the prompt project needs to be completed manually, which is not friendly enough for non-AI professionals or designers who do not understand AI; the other is that the generated 3D assets often do not meet industry standards and cannot be put into use no matter how beautiful they are.

In view of these two points, ChatAvatar has given two specific and effective solutions.

On the one hand, ChatAvatar realizes a second path besides manual input prompt engineering, and is also a shortcut more suitable for ordinary people: describing needs through direct dialogue through "Party A mode".

The team’s official Twitter said that in order to realize this feature, ChatAvatar developed a method of converting conversational descriptions into portrait features based on GPT’s capabilities.

Designers only need to keep chatting with GPT and describe the "feeling" they want:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

GPT can automatically help complete the prompt project and display the results Delivered to AI:

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

# In other words, if ControlNet is the "Game Changer" for the 2D industry, then for the 3D industry, ChatAvatar can convert text into 3D , is nothing short of a game changer for the industry.

On the other hand, what is more important is that ChatAvatar is perfectly compatible with the CG pipeline, that is, the generated assets meet industry requirements in terms of topology, controllability and speed.

This not only means that after generating 3D assets, the downloaded content can be directly imported into various post-production software for secondary editing, with greater controllability;

At the same time, the generated Models and high-precision material maps can also achieve extremely realistic rendering effects in later rendering.

In order to achieve such an effect, the team developed a progressive 3D generation framework DreamFace for ChatAvatar.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

The key lies in the underlying data used to train the model, which is the world's first large-scale, High-precision, multi-expression face high-precision data set.

Based on this data set, DreamFace can efficiently complete the generation of

product-level three-dimensional assets, that is, the generated assets have regular topology, materials, and bindings.

DreamFace mainly includes three modules: geometry generation, physics-based material diffusion and animation capability generation.

By introducing an external 3D database, DreamFace can directly output assets that comply with the CG process.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

△The effect of generated asset-driven rendering

The essence of solving the above two major technical bottlenecks The trend of the times has been further accelerated by the AIGC torrent, and "generation" will replace "search" -

The Shadow Eye team believes that "generation" will become the way to obtain a new generation of digital assets.

Previously, when we needed to find a picture or asset that met our needs, we usually used search engines to query.

The huge "search box" and neat asset cards displayed on the homepage of the ChatAvatar project look like a search engine, but in fact it is a completely different way of finding assets than search.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

△ChatAvatar project homepage

Yingmu Technology CTO Zhang Qixuan introduced this:

In the past, if we needed an illustration, maybe You have to search repeatedly in multiple libraries, or use more complicated methods such as Photoshop synthesis and hand-painting to get the results. But after the emergence of technologies such as Stable Diffusion, you only need to describe the desired image through text, and you can directly generate results that meet your needs.

This is a huge impact on traditional asset libraries. The goal of ChatAvatar is to replace the traditional search-based 3D asset library with 3D generation.

The next frontier in the field of AIGC

ChatGPT has stirred up waves with one stone. After entering the AI ​​2.0 era, people’s attention has also turned to multi-modal information including images, videos, 3D and other information. AI.

As far as the field of 3D generation is concerned, whether it is the film, television or game industries, the 3D content production and consumption market is already large enough, but it is hampered by technical difficulties at the production level.

For example, Transformer, which is very popular in the field of text, has relatively limited use in the field of 3D generation.

Last summer, when the

文生图 field achieved results due to Diffusion Model, people began to expect Text generation 3D to have the same amazing performance. Once the 3D creation technology of generative AI matures, content creation such as VR and video will take off.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

△"Van Gogh Wind Photography" generated by diffusion model Midjourney5.1

In fact, Both technology giants and start-up companies are indeed secretly working in the direction of Text-to-3D.

In September last year, Google released FreamFusion, which generates 3D models based on text prompts, claiming that it does not require 3D training data or modify the image diffusion model. Following closely, Meta also launched the Make-A-Video model that can generate videos from text with one click.

Later in the Text-to-3D AI model team, Nvidia Magic3D, OpenAI’s latest open source project Shap-E, etc. have appeared successively. The top computer graphics conference SIGGRAPH 2023 will be held in August this year. There are also many papers related to Text-to-3D.

Yingmo Technology’s paper on DreamFace, a text-guided progressive 3D generation framework, is one of them.

ChatAvatar is also the most generative model product focused on 3D digital human assets so far.

The AI ​​startup company behind itYingmu Technology was incubated from the MARS Laboratory of Shanghai University of Science and Technology in 2020. After its establishment, it received two rounds of investment from Qiji Chuangtan and Sequoia Seeds.

The company focuses on the research and productization of computer graphics and generative AI. In 2021, before AIGC made huge waves, the company had already launched Wand, the first AIGC ToC painting application in China, and the product once topped the AppStore partition.

单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

And this forward-looking team, which is already well-known in the industry, average age is only 25 years old.

After specifically anchoring the first commercialization scenario on digital people, ChatAvatar is their latest progress in this direction by taking advantage of AIGC.

As a newly launched product, ChatAvatar has exceeded the expectations of the Yingmo team in terms of product effects such as compatibility, completion and accuracy. However, in Wu Di's words, the process of getting here was "very embarrassing."

The main reason is nothing more than "lack of people". At present, Shadow Eye has made progress in multi-category 3D generation technology, and the next step is to launch "3D generated large models".


单卡30秒跑出虚拟3D老婆!Text to 3D生成看清毛孔细节的高精度数字人,无缝衔接Maya、Unity等制作工具

##△Yingmo Technology will launch its first multi-modal model in May The cross-platform 3D search engine Rodin connects multiple 3D asset platforms such as Sketchfab, and supports searching for 3D through text, 3D through pictures, and even 3D through 3D. The search engine is just the primary form of Rodin, and Shadow Eye will build Rodin into a large 3D generated model.

To continue to move forward, more engineering teams, technical artists, and product talents who embrace generative AI are needed to join the team. As a team with R&D as its main background, such talents are still in short supply.

"People are the measure of all things," Wu Di said, "We need more like-minded people to join in and jointly promote innovative development in the 3D field."

As you can see, the technology behind ChatAvatar Building from scratch reveals the continuous innovation of an AI start-up company; and the company's desire for talents from small to large reveals that under the wave of AIGC, every segment wants to emerge from the water Heart on the water.

Are you willing to embrace generative AI and become a Game Changer in the Text-to-3D field?

The above is the detailed content of Get a virtual 3D wife in 30 seconds with a single card! Text to 3D generates a high-precision digital human with clear pore details, seamlessly connecting with Maya, Unity and other production tools. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks ago By 尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
1 months ago By 尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks ago By 尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

ChatGPT now allows free users to generate images by using DALL-E 3 with a daily limit ChatGPT now allows free users to generate images by using DALL-E 3 with a daily limit Aug 09, 2024 pm 09:37 PM

DALL-E 3 was officially introduced in September of 2023 as a vastly improved model than its predecessor. It is considered one of the best AI image generators to date, capable of creating images with intricate detail. However, at launch, it was exclus

Why is Gaussian Splatting so popular in autonomous driving that NeRF is starting to be abandoned? Why is Gaussian Splatting so popular in autonomous driving that NeRF is starting to be abandoned? Jan 17, 2024 pm 02:57 PM

Written above & the author’s personal understanding Three-dimensional Gaussiansplatting (3DGS) is a transformative technology that has emerged in the fields of explicit radiation fields and computer graphics in recent years. This innovative method is characterized by the use of millions of 3D Gaussians, which is very different from the neural radiation field (NeRF) method, which mainly uses an implicit coordinate-based model to map spatial coordinates to pixel values. With its explicit scene representation and differentiable rendering algorithms, 3DGS not only guarantees real-time rendering capabilities, but also introduces an unprecedented level of control and scene editing. This positions 3DGS as a potential game-changer for next-generation 3D reconstruction and representation. To this end, we provide a systematic overview of the latest developments and concerns in the field of 3DGS for the first time.

CLIP-BEVFormer: Explicitly supervise the BEVFormer structure to improve long-tail detection performance CLIP-BEVFormer: Explicitly supervise the BEVFormer structure to improve long-tail detection performance Mar 26, 2024 pm 12:41 PM

Written above & the author’s personal understanding: At present, in the entire autonomous driving system, the perception module plays a vital role. The autonomous vehicle driving on the road can only obtain accurate perception results through the perception module. The downstream regulation and control module in the autonomous driving system makes timely and correct judgments and behavioral decisions. Currently, cars with autonomous driving functions are usually equipped with a variety of data information sensors including surround-view camera sensors, lidar sensors, and millimeter-wave radar sensors to collect information in different modalities to achieve accurate perception tasks. The BEV perception algorithm based on pure vision is favored by the industry because of its low hardware cost and easy deployment, and its output results can be easily applied to various downstream tasks.

Choose camera or lidar? A recent review on achieving robust 3D object detection Choose camera or lidar? A recent review on achieving robust 3D object detection Jan 26, 2024 am 11:18 AM

0.Written in front&& Personal understanding that autonomous driving systems rely on advanced perception, decision-making and control technologies, by using various sensors (such as cameras, lidar, radar, etc.) to perceive the surrounding environment, and using algorithms and models for real-time analysis and decision-making. This enables vehicles to recognize road signs, detect and track other vehicles, predict pedestrian behavior, etc., thereby safely operating and adapting to complex traffic environments. This technology is currently attracting widespread attention and is considered an important development area in the future of transportation. one. But what makes autonomous driving difficult is figuring out how to make the car understand what's going on around it. This requires that the three-dimensional object detection algorithm in the autonomous driving system can accurately perceive and describe objects in the surrounding environment, including their locations,

The perfect combination of ChatGPT and Python: creating an intelligent customer service chatbot The perfect combination of ChatGPT and Python: creating an intelligent customer service chatbot Oct 27, 2023 pm 06:00 PM

The perfect combination of ChatGPT and Python: Creating an Intelligent Customer Service Chatbot Introduction: In today’s information age, intelligent customer service systems have become an important communication tool between enterprises and customers. In order to provide a better customer service experience, many companies have begun to turn to chatbots to complete tasks such as customer consultation and question answering. In this article, we will introduce how to use OpenAI’s powerful model ChatGPT and Python language to create an intelligent customer service chatbot to improve

Can chatgpt be used in China? Can chatgpt be used in China? Mar 05, 2024 pm 03:05 PM

chatgpt can be used in China, but cannot be registered, nor in Hong Kong and Macao. If users want to register, they can use a foreign mobile phone number to register. Note that during the registration process, the network environment must be switched to a foreign IP.

How to install chatgpt on mobile phone How to install chatgpt on mobile phone Mar 05, 2024 pm 02:31 PM

Installation steps: 1. Download the ChatGTP software from the ChatGTP official website or mobile store; 2. After opening it, in the settings interface, select the language as Chinese; 3. In the game interface, select human-machine game and set the Chinese spectrum; 4 . After starting, enter commands in the chat window to interact with the software.

How to develop an intelligent chatbot using ChatGPT and Java How to develop an intelligent chatbot using ChatGPT and Java Oct 28, 2023 am 08:54 AM

In this article, we will introduce how to develop intelligent chatbots using ChatGPT and Java, and provide some specific code examples. ChatGPT is the latest version of the Generative Pre-training Transformer developed by OpenAI, a neural network-based artificial intelligence technology that can understand natural language and generate human-like text. Using ChatGPT we can easily create adaptive chats

See all articles