又一Sora級選手來炸街！我們拿它和Sora、可靈PK了下

當Sora「千呼萬喚」不出來時，OpenAI 的對手們卻紛紛祭出大殺器來炸街。

Sora will really be stolen if it is not open for use!

Today, San Francisco startup Luma AI played a trump card and launched a new generation of AI video generation model Dream Machine. Free and available to everyone.

According to reports, this model can generate high-quality, realistic videos based on simple text descriptions, with effects comparable to Sora.

As soon as the news came out, a large number of users crowded into the official website to try it out.

Although the official claims that the model can generate 120 frames of video in just two minutes, many users have been waiting for hours on the official website due to the surge in traffic.

Barkley Dai, Luma’s head of product growth, had to post on Discord to explain -

"We are currently facing huge demand and are working hard to increase our processing capabilities. All video generation tasks will be retained, just as needed Wait in the queue for a while. Once we increase the processing capacity, I will let you know right here! "

How effective is the Dream Machine?

Some netizens said that Luma is currently the new king in the field of AI video.

Some netizens said, "We no longer need Sora!" I wonder what OpenAI thought after seeing this.

However, some netizens complained that after making 8 videos, the system prompted "maximum usage limit exceeded" and did not explain how long they should wait before making new videos.

Netizens are going crazy

In the past few days, the AI ​​video circle has gone crazy. You can sing and I will appear.

First, Kuaishou Keling opened a closed beta, with more than 50,000 people queuing up. Then Luma launched its killer feature, Dream Machine, which is free for everyone.

Compared with other AI video models, Dream Machine has the following characteristics:

1. Fast, 120 frames can be generated in 120 seconds;

2. The action is realistic, smooth, and integrated Movie-level photography skills and dramatic tension;

3. Strong character consistency and the ability to simulate the physical world;

4. Natural camera movements that can match the emotions of the scene.

Luma officials and netizens have worked together one after another to present a wonderful visual feast.

For example, this text-generated video shows a car racing down the road. Whether it’s driving or camera transitions, everything is smooth and lifelike. A camera low to the ground tracks a group of small hamsters deep into their burrows. This scene is similar to Sora’s ant video, but Dream Machine uses a Tusheng video function, commonly known as “cushion”.

A bald man wearing an orange T-shirt moves around the room. The lifelikeness of the character and the composition of the picture are comparable to those of blockbusters. This is a shot of a ruins scene. The discarded ropes, wooden boards on the ground and the graffiti on the walls appear natural and realistic.In the picture, a young woman is dancing with her skirt waving, her movements are smooth and fluid, just like a luxury advertising blockbuster. However, the only drawback is that the skirt and hair will deform. Some netizens even generated an action scene of a killer gunfight. Netizen @ai_mov_director also used it to generate a 1-minute feature film - "Break The Tie". In terms of maintaining character consistency, Dream Machine has two brushes.

In addition to generating realistic videos, Dream Machine can also try different styles.

For example, Japanese anime style: Disney style:

Overall, Dream Machine is commendable in terms of video fidelity and smoothness, but it is not perfect.

Julien Vallee, who has directed commercials for Apple, Samsung, Google and other well-known brands, said that Dream Machine can imitate natural camera movements, especially when shooting handheld, the effect is very realistic. However, like other models, it requires some trial and error to produce great shots.

Vincent Video Circle Battle

2024 is an election year, and OpenAI has been hiding Sora in order not to cause trouble.

When Sora's "thousands of calls" didn't come out, the opponents used their big weapons to destroy the streets.

The AI ​​video field is undergoing a rapid change.

Since both Dream Machine and Keling are under the banner of "competing against Sora", then we simply set up an arena and let Dream Machine, Keling and Sora compete on the same stage.

Prompt 1:photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee.

Sora:

Dream Machine:

Keling:

Prompt 2: Nighttime footage of hermit crabs using light bulbs as shells.

Chinese prompt word 2: Sojourn Night shot of a crab using a light bulb as a shell.

Sora:

Dream Machine:

Keling:

Prompt 3: macro shot of a leaf showing tiny trains moving through its veins.

Sora：

Dream Machine：

Keling：

Prompt 4: A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. She wears sunglasses and red lipstick. She walks confidently and casually. The street is damp and reflective, creating a mirror effect of the colorful lights. Many pedestrians walk about.

Chinese Prompt Word 4: A fashionable woman walks on the streets of Tokyo, which are full of warm neon lights and vivid city signs. She was wearing a black leather jacket, a long red skirt, black boots, and was holding a black purse. She wears sunglasses and red lipstick. She walked with confidence and ease. The streets are wet and reflective, creating a mirror effect of colored lights. Many pedestrians were walking around.

Sora：

Dream Machine：

Keling：

Prompt 5:Archeologists discover a generic plastic chair in the desert, excavating and dusting it with great care.

Chinese Tip 5: Archaeologists found an ordinary plastic chair in the desert. They carefully dug it up and dusted it off.

Sora:

Dream Machine:

Keling:

Prompt 6: a computer hacker labrador retreiver wearing a black hooded sweatshirt sitting in front of the computer with the glare of the screen emanating on the dog's face as he types very quickly.

Chinese Prompt Word 6: A computer hacker Labrador retriever wearing a black hooded sweatshirt sits in front of a computer and as he types quickly, the screen The glare hits the dog's face.

Sora:

Dream Machine:

Keling:

What is the origin of this company led by NVIDIA?

Dream Machine has become popular, and the company behind it, Luma AI, has also stolen the limelight.

Luma AI was founded in 2021 and was initially a technology company focused on 3D content generation.

CEO Amit Jain was a computer vision system engineer at Apple, and CTO Alex Yu was a graduate student at the University of California, Berkeley (he gave up his PhD to start Luma AI). The two have made achievements in the fields of 3D vision, machine learning, real-time graphics and other fields.

It is reported that this company has gone through several rounds of financing.

The Series A financing was led by Amplify Partners, Nventures (Nvidia’s investment arm) and General Catalyst, raising a total of US$20 million; the Series B financing was led by Silicon Valley’s top venture capital firms Andreessen Horowitz and Nvidia, raising US$43 million Dollar. To date, the company has raised more than $70 million in funding and is valued at between $200 million and $300 million.

Last November, Luma AI launched Vincent 3D model Genie on the Discord server. Later, version 1.0 was launched, which improved the drawing time from more than 20 seconds to less than 10 seconds.

Unexpectedly, this time Luma AI directly switched to the field of AI video.

According to the official website, the Luma AI core team has only 34 people, and judging from the names, 5 of them are Chinese.

Co-founder and CTO Alex Yu, graduated from the University of California, Berkeley in 2021. During this period, he conducted NeRFs research with Professor Angjoo Kanazawa at the Berkeley Artificial Intelligence Research Laboratory.

Angela Dong graduated from the University of California, Berkeley in the same year. She interned at companies such as Drive.ai, Lyft Level 5 and Zipline, and then joined Cruise as a simulation engineer, focusing on creating synthetic data for perception model training. Currently, she works as a Machine Learning Engineer at Luma.

Chief Scientist Jiaming Song graduated from Tsinghua University with a bachelor’s degree and a master’s and doctoral degree from Stanford University. Before joining Luma AI, he served as a research scientist in NVIDIA's Learning and Perception research team and Deep Imagination research team.

Additionally, Quei-An Chen and Paul Yoo serve as research scientists at Luma.

(Quei-An Chen on the left, Paul Yoo on the right)

Among them, Quei-An Chen is deeply involved in the NeRF field and has become famous for launching many popular open source projects on Github. Such as Neural Scene Flow Fields and Instant-NGP. Before joining Luma, he participated in multiple 3D vision technology projects at DENSO and LINE.

^Link:

^{https://lumalabs.ai/dream-machine/creations}

以上是又一Sora級選手來炸街！我們拿它和Sora、可靈PK了下的詳細內容。更多資訊請關注PHP中文網其他相關文章！