The effect is explosive! OpenAI's first video generation model released, smooth and high-definition in 1 minute, netizens: The entire industry is RIP

Just now, Ultraman released OpenAI’s first video generation modelSora.

Perfectly inherits the image quality and command-following capabilities of DALL·E 3, and can generate high-definition videos up to 1 minute long.

#AI imagined the Spring Festival of the Year of the Dragon, with red flags waving and huge crowds of people.

Many children watched the dragon dance team curiously, and some people took out their mobile phones to record people's different behaviors.

The streets of Tokyo after the rain, the wet groundReflectionThe neon light and shadow effects are comparable to RTX ON.

# The window of the moving train is occasionally blocked, and the reflection of the characters in the car briefly appears, which is very stunning.

You can also watch a Hollywood blockbuster-like movie trailer:

Vertical screen super close-up perspective Below, this lizard is full of details:

Netizens called the game over and lost their jobs:

Some people have even begun to "mourn" an entire industry:

AI understands the physical world in motion

OpenAI stated that it is teaching AI understands and simulates the physical world in motion, with the goal of training models to help people solve problems that require real-world interaction

Generating videos based on text prompts is just one step in the entire plan.

Currently Sora can generate complex scenes with multiple characters and specific movements. It can not only understand the user's prompts The requirements presented in , also understand how these objects exist in the physical world.

Sora can also create multiple shots within a single video and rely on a deep understanding of language to accurately interpret cue words, preserving character and visual style.

Beautiful, snowy Tokyo is bustling with people. The camera moves through bustling city streets, following several people enjoying a beautiful snowy day and shopping at nearby stalls. The gorgeous cherry blossom petals flutter in the wind along with the snowflakes.

OpenAI is not shy about Sora's current weaknesses, pointing out that it may have difficulty accurately simulating the physical principles of complex scenes, and may not be able to understand cause-and-effect relationships.

For example, "Five gray wolf cubs were playing and chasing each other on a remote gravel road." The number of wolves will change, and some will appear or disappear out of thin air.

The model may also obfuscate the spatial details of the cue , such as confusing left and right, and may be difficult to Precisely describe events that occur over time, such as following a specific camera trajectory. For example, in the prompt word "The basketball passes through the basket and then explodes", the basketball is not blocked by the basket correctly.

In terms of technology, OpenAI has not disclosed much at present. A brief introduction is as follows:

Sora is a diffusion model, starting from noise, it can generate the entire video at once or extend the length of the video,

The key is that Generate predictions for multiple frames at once, ensuring that the subject of the picture remains unchanged even if it temporarily leaves the field of view.

Similar to the GPT model, Sora uses the Transformer architecture, which is highly scalable.

In terms of data, OpenAI represents videos and images as patches, similar to tokens in GPT.

With this unified data representation, models can be trained on a wider range of visual data than before, covering different sustained Time, resolution and aspect ratio.

Sora is built on past research on the DALL·E and GPT models. It uses DALL·E 3's restatement prompt word technology to generate highly descriptive annotations for visual training data, so it can more faithfully follow the user's textual instructions.

In addition to being able to generate videos based solely on textual instructions, the model is also able to take existing static images and generate videos from them, accurately animate the image content and pay attention to small details.

The model can also take existing video and expand it or fill in missing frames, see the technical paper for more information (to be released later) .

Sora is the foundation for models that can understand and simulate the real world. OpenAI believes that this feature will become an important milestone in achieving AGI.

Ultraman takes orders online

Currently, some visual artists, designers and filmmakers (as well as OpenAI employees) have gained access to Sora.

They began to publish new works continuously, and Ultraman also started taking orders online.

Bring your prompt word @sama, and you may receive a generated video reply.

The above is the detailed content of The effect is explosive! OpenAI's first video generation model released, smooth and high-definition in 1 minute, netizens: The entire industry is RIP. For more information, please follow other related articles on the PHP Chinese website!