The new blood of virtual reality, the 3D modeling industry empowered by AI

What is the upper limit of generative AI? The answer to this question may not be available in a short time, at least now generative AI seems to have conquered a new field. Previously, the scope of work of generative AI was mainly focused on word processing, painting, sound processing, etc., but the capabilities of generative AI are obviously much more than that.

Recently, the open source generative AI company Stability AI officially released the Stable Animation SDK, and the much-anticipated Stable Diffusion model (hereinafter referred to as Stable) also officially released a new version. Now users can better control the 3D models generated by AI. , and modify specific parameters.

Perhaps what many people think after seeing this news is: "3D model? Does it mean squares, strips and the like?" After all, in the eyes of most people, the real complexity of 3D modeling is probably beyond the capabilities of ordinary AI. processed. However, here lies the charm of generative AI. Through data processing and analysis, coupled with the understanding of natural language, today's AI can generate more complex and complete models based on descriptions, and is not limited to simple block models.

Not just 2D to 3D

Prior to this, Stable has attracted a lot of attention because it can directly convert 2D pictures and descriptions into 3D models. Although the 3D models generated by Stable are not as precise as those that professionals spend a long time building, they are still Considering the time required to generate it, it is enough to give everyone a big shock.

In the latest demonstration video released by Stability AI, Stable’s 3D models are no longer limited to still life. Even characters that have been moving can be easily transformed into 3D models, and the range and posture of the movements are very similar to the original ones. near. Similar technology is often used in animation production. In order to make the pictures and objects appear more three-dimensional, some animations will convert the picture from 2D to 3D to highlight the tension of the picture.

Source: Stability AI

In the traditional animation industry, converting 2D images into 3D requires many staff to be busy for a period of time. With the help of Stable, you only need to input 2D images into the model to obtain high-quality The starting 3D model greatly reduces the time and cost required for modeling.

Of course, if this is the case, perhaps practitioners will be happy. For the majority of netizens, what is the use of Stable? The key is that Stable's 3D model generation does not require detailed guidance. Even if it is just a simple drawing, Stable can generate a 3D model that is almost the same. For example, this graffiti looks like a kindergarten child, after being "polished" by Stable. It becomes a fairly watchable 3D picture.

Source: YouTube

Stable's desire and pursuit is to make all your paintings and texts come to life. Therefore, all results of Stability AI are directly disclosed and provided to netizens in an open source manner. For the majority of two-dimensional enthusiasts, perhaps this is the easiest way to get their "paper wives" moving.

And from Stable’s 3D dynamic model generation capabilities, we can also see some future application scenarios, such as cheaper and more convenient motion capture systems. In theory, as long as the computing power is sufficient, the images captured by the camera can be captured in real time. Generate corresponding 3D model actions.

What other wonderful uses are there besides this? I don’t know if you have seen a recent hot news. Caryn Marjorie, an overseas Internet celebrity, used GPT-4 to copy a digital version of herself by working with an AI team, and then sold the right to use the digital version for one dollar per minute. Sell ​​to your own fans.

In just one week, Caryn Marjorie earned $71,000 from this service, and all she provided was voice chat chat services. As visual creatures, our sensitivity to sound is actually lower than that of images. If Stable is also applied to related fields, is it possible to create a true AI girlfriend? It's movable and conversational, enough to soothe your empty heart.

Ahem, okay, let’s stop this topic for now. At least with the current model efficiency and computing power scale, it is probably very difficult for individuals to achieve real-time and high-specification 3D dynamic model generation, but considering the semiconductor industry With the speed of progress, perhaps this day is not far away from us.

New productivity tools

Stable The biggest problem before was that 3D models could only be generated based on descriptions or images. If the generated effect was not good, the images or text information could only be re-adjusted to regenerate. Depending on the performance of the graphics card, the generation time of the 3D model There will also be differences. Compared with traditional question-and-answer AI such as ChatGPT, the time cost of Stable is much higher.

So, although Stable's 3D model generation effect is far better than similar applications in the past, the usage scenarios are very limited, and it can only provide community enthusiasts with a simple and convenient 3D model generation tool. Community users have long hoped that Stability AI can add parameter adjustment functions to Stabel, so that unsatisfactory model details can be modified.

The response given by Stability AI is Stable Animation SDK. This interface can be loaded into Stabel's model. After using Stabel to generate a 3D model, users can directly input the corresponding parameters through the interface to adjust or add model details. Make the model more in line with user requirements.

Judging from the description file of the interface, there are many parameters that support modification, ranging from basic color, shape, size, texture to action posture, etc., and the adjustment process does not require the input of professional data or nouns. You only need to enter text information as shown in the figure when generating a 3D model.

For example, if you generate a 3D model of a puppy, and then feel that the pattern on the puppy is not satisfactory, you only need to enter the pattern description you want from the interface, and Stabel will modify the model and re-render it based on the description. Related layers.

In addition, Stable Animation SDK also supports the input of action commands, which allows the static 3D model to directly execute your action commands. For example, you render a flying dragon and then enter the command "Let the dragon fly and breathe fire" , Stable will start rendering the 3D model in action.

Moreover, Stable also provides photography functions. Users can adjust a series of parameters such as camera position, lighting effects, and background to record static and dynamic videos of 3D models. Yes, everyone has probably guessed that 3D modeling, 3D animation and other 3D modeling-related industries will feel the "warmth" from AI.

Some netizens believe that combining the Stable platform and virtual reality equipment may bring about a dramatic improvement in the productivity of virtual reality equipment. Everyone should have seen the Marvel movie "Iron Man". The protagonist Stark in the movie has an advanced artificial intelligence program "Jarvis", which gave Stark a lot of help when he made the Iron Man armor.

One of the clips is that Stark directly generated a 3D model of a part through dialogue, then adjusted it and applied it to the armor. Does this process sound familiar? Yes, in a sense, this is the future version of Stable ChatGPT. 3D models are created directly through dialogue, allowing designers to directly inspect the appearance and usage effects of items in virtual reality devices.

Putting this process into real-life photos is equivalent to simplifying the most time-consuming proofing and adjustment process in product design, and substantially improving the efficiency of the entire process from product design to implementation. In addition, designers can use and experience their products in advance by taking advantage of the capabilities of virtual reality devices.

Of course, in the current product design process, similar 3D model software has been widely used to render scenes. However, the advantage of Stable lies in the speed of generation. Models that originally took hours or even days to adjust and render are now only It takes one-tenth or less time to generate, and the efficiency improvement behind it is self-evident.

With the proliferation of generative AI, we can see that AI will have a profound impact on our society, production and other aspects. Today is a 3D model, what will it be tomorrow? I am very excited.

Source: Lei Technology Ieitech

The above is the detailed content of The new blood of virtual reality, the 3D modeling industry empowered by AI. For more information, please follow other related articles on the PHP Chinese website!