Home > Technology peripherals > AI > Stable Video Diffusion is here! 3D synthesis function attracts attention, netizens: progress is too fast

Stable Video Diffusion is here! 3D synthesis function attracts attention, netizens: progress is too fast

PHPz
Release: 2023-11-22 20:14:48
forward
709 people have browsed it

Stable Video Diffusion officially started to process videos -

Released the generative video modelStable Video Diffusion (SVD).

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Stability AI official blog shows that the new SVD supports text-to-video and image-to-video generation:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

and also Supports the transformation of objects from a single perspective to multiple perspectives, that is, 3D synthesis:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

According to external evaluation, the official claims that SVD is even better than runway and Pika. Video generation AI is more popular among users.

Although only the basic model has been released so far, the official revealed that "it is planning to continue to expand and establish an ecosystem similar to stable diffusion"

The paper code weight is now online.

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Recently, new methods of play have been emerging in the field of video generation. Now it is the turn of Stable Diffusion to appear, so that netizens have lamented "fast", such progress is too fast. !

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

But judging from the demo effect alone, more netizens said they were not very surprised.

Although I like SD, and these demos are great...but there are also some flaws, the lighting and shadow are wrong, and the overall incoherence(video flickers between frames).

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

All in all, this is the beginning. Netizens are very optimistic about SVD’s 3D synthesis function:

I can guarantee that there will be more soon. When good things come out, you only need a brief description to present a complete 3D scene

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

SD video official version is coming

In addition to what is shown above Yes, the official has also released more demonstrations, let’s take a look first:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Space walks are also arranged:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

You can also keep the background still and only let the two birds move:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

The research paper on SVD has also been released. According to reports, SVD is based on Stable Diffusion 2.1 and uses about The base model is pre-trained on a video data set of 600 million samples.

Easily adaptable to a variety of downstream tasks, including multi-view synthesis from a single image by fine-tuning multi-view datasets.

After fine-tuning, two image-to-video models were officially announced. These models can generate 14-frame (SVD) and 25-frame (SVD-XT) video at custom frame rates from 3 to 30 frames per second depending on the user's needs

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

#After fine-tuning the multi-view video generation model, we named it SVD-MV

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

According to the test results, on the GSO dataset, SVD-MV scored excellent For multi-view generation models Zero123, Zero123XL, SyncDreamer:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

It is worth mentioning that Stability AI stated that SVD is currently limited to research and is not suitable for practical or commercial applications. SVD is not currently available to everyone, but user waiting list registration is open.

The explosion of video generation

Recently, there has been a state of "melee" in the field of video generation

There was previously Vincent Video AI developed by PikaLabs:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Later, the so-called "most powerful video generation AI in historyMoonvalley was launched:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Recently, Gen-2's "Motion Brush" function has also been officially launched. You can draw where you want:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Now SVD has appeared again , and there is the possibility of 3D video generation.

However, there seems to be not much progress in text to 3D generation, and netizens are also very confused about this phenomenon.

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Some people think that data is the bottleneck that hinders development:

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Some netizens think that the problem is that the ability of reinforcement learning is not strong enough

Stable Video Diffusion问世!3D合成功能引关注,网友:进步太快

Do you know the latest progress in this area? Welcome to share in the comment area~

Paper link: https://static1.squarespace.com/static/6213c340453c3f502425776e /t/655ce779b9d47d342a93c890/1700587395994/stable_video_diffusion.pdf What needs to be rewritten is:

The above is the detailed content of Stable Video Diffusion is here! 3D synthesis function attracts attention, netizens: progress is too fast. For more information, please follow other related articles on the PHP Chinese website!

Related labels:
source:51cto.com
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template