Home Technology peripherals AI DAMO Academy releases one-stop AI video creation platform 'Xunguang' to create a new AI workflow

DAMO Academy releases one-stop AI video creation platform 'Xunguang' to create a new AI workflow

Jul 12, 2024 pm 04:24 PM
Bodhidharma Academy industry World Artificial Intelligence Conference Seek light

This year is the first year of the explosion of AI video generation, and algorithm models and product applications represented by Sora are constantly emerging. In just a few months, we’ve seen the launch of dozens of video generation tools, and AI-based video creation is starting to take off.

But new technologies also cause more challenges and doubts. In addition to the well-known "blind box opening" phenomenon, video content generated by AI has also been frequently criticized for its poor controllability and cumbersome processing workflow.

OpenAI once invited a professional video production team to test Sora. Among them, the Shy Kids team from Toronto used Sora to produce a short film with a balloon man theme, which perfectly combined creativity and AI technology. People are impressed.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

In fact, the entire short film is not the result of Sora’s direct output, but consists of multiple video clips. When Sora generates different videos, it is difficult to ensure the consistency of the protagonist. Therefore, after introducing a lot of manual post-editing, they presented the final short film effect. The creators of Shy Kids concluded, "Sora's technology is cool, but its generation process is difficult to control."

Accurate control of generated content is an important requirement in AI video creation, and it is also what today's algorithms face a big challenge.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

To this end, at the just-concluded Shanghai World Artificial Intelligence Conference (WAIC), DAMO Academy released the one-stop AI video creation platform "Xunguang".

It is positioned as PUGC’s one-stop AI video creation platform, which can assist users in creating scripts, storyboards, etc., and improve the efficiency of the entire creative process through workflow integration, supporting rich AI for generating and uploading materials. Editing provides more than ten AI editing functions such as character control, scene control, style transfer, camera movement control, target addition/elimination/modification, etc., allowing elements and objects in the video to be accurately controlled.

Dharma Academy hopes to further improve the efficiency of AI video creation through the Xunguang platform. The goal is to use AI capabilities to reshape the entire process of traditional video production and create a new video workflow in the AI ​​era.

First time in the industry
Layer-based video editing

In the early stages of Xunguang’s research and development, DAMO Academy also conducted extensive and intensive research with film and television media practitioners and creators , to understand their needs and pain points for video AIGC creation. They found that video layers were the most frequently mentioned and most urgent need among almost all video creators.

Based on this, Xunguang Platform has launched a systematic video layer editing function for the first time in the industry. By inputting text, users can generate a video that matches the text description and has a transparent background, and blend it into other background videos with one click. Based on the traditional video generation capabilities, content is generated in a more flexible form such as layers.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

Xunguang also provides the layer disassembly function. With a single tap, the selected target will be immediately disassembled into separate layer videos, and then different background videos can be embedded smoothly.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

Users can fuse different foreground layers with different backgrounds to combine more new videos. The ability of layer fusion further stimulates AI creativity and imagination, while maintaining the consistency of scenes and characters between multiple shots.
DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow
In the view of DAMO Academy, AI will not replace the work of creators, but will optimize the workflow of video creation and become a new engine driven by creativity.

One-stop AI creation platform
Simpler interaction, richer editing capabilities

Script creation, storyboard design, material editing... The traditional video creation steps have a clear division of labor , The cycle is lengthy. With the support of AI technology, the creative steps that were originally scattered in different production processes can now be completed smoothly on the light-finding platform.

"We hope to make video editing as simple, intuitive and easy to use as operating ppt." Chen Weihua, a senior algorithm expert at the Visual Technology Laboratory of Damo Academy, introduced at the scene that a major highlight of the light-seeking platform is the interaction aspect.

The Xunguang platform fully takes into account the characteristics of AI video creation when designing, abstracting each video project into multiple sub-shots. Users can automatically generate a group of sub-shots based on the script, or they can upload the original video materials themselves. , divided into multiple sub-shots by the algorithm.

In the creative space, users can easily view each shot. Multiple shots within a scene can be collapsed or expanded. The order between scenes can be adjusted by dragging. Shots within a scene can also be Drag and drop. Users can also add and create new sub-shots at any location, call image generation or video generation capabilities to generate content, or add various existing materials.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

For each shot, Xunguang provides complete and intelligent AI video editing capabilities for processing, which can be edited at the semantic level rather than the pixel level based on user intentions. Any local targets such as the human body, face, foreground, and background in the split shot can be finely edited and modified.

For example, understanding the camera movement control of spatial depth of field;

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

Another example, being able to understand the target elimination/modification of the relative relationship between objects.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

In terms of editing global elements of the video, the light-finding platform provides more than 20 style migrations.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

Xunguang also provides practical video editing functions such as frame rate control and video super-resolution.

DAMO Academy releases one-stop AI video creation platform Xunguang to create a new AI workflow

“We hope that all elements in a video can be edited and modified, so as to provide users with the greatest freedom in creation,” said Chen Weihua.

Written at the end

Today, we are in the midst of a wave of change in AIGC, and AI has the potential to give birth to new video workflows. Both professional film and television practitioners and UGC users who love creation will benefit from it.

"If you want to do your job well, you must first sharpen your tools." Damo Academy hopes that the Xunguang video creation platform can become an exclusive video studio for every creator, achieving a closer relationship between AI and creators , efficient collaboration, truly unleashing the productivity of AI.

To this end, DAMO Academy Vision Technology Laboratory has made a lot of technical reserves. The laboratory is committed to the research of multi-modal visual signal understanding and generation technology. The current key research directions include more accurate image/video/3D content generation, more controllable image/video/3D content editing, and more efficient generation. Frames, multimodal understanding - generative frames, etc.

Chen Weihua said that "Xunguang" will be launched for internal testing in the near future and will continue to iterate and optimize interactions. Creators are welcome to customize their own AI workflow.

Internal beta application address:
https://xunguang.damo-vision.com/

The above is the detailed content of DAMO Academy releases one-stop AI video creation platform 'Xunguang' to create a new AI workflow. For more information, please follow other related articles on the PHP Chinese website!

Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

SublimeText3 Chinese version

SublimeText3 Chinese version

Chinese version, very easy to use

Zend Studio 13.0.1

Zend Studio 13.0.1

Powerful PHP integrated development environment

Dreamweaver CS6

Dreamweaver CS6

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

DeepMind robot plays table tennis, and its forehand and backhand slip into the air, completely defeating human beginners DeepMind robot plays table tennis, and its forehand and backhand slip into the air, completely defeating human beginners Aug 09, 2024 pm 04:01 PM

But maybe he can’t defeat the old man in the park? The Paris Olympic Games are in full swing, and table tennis has attracted much attention. At the same time, robots have also made new breakthroughs in playing table tennis. Just now, DeepMind proposed the first learning robot agent that can reach the level of human amateur players in competitive table tennis. Paper address: https://arxiv.org/pdf/2408.03906 How good is the DeepMind robot at playing table tennis? Probably on par with human amateur players: both forehand and backhand: the opponent uses a variety of playing styles, and the robot can also withstand: receiving serves with different spins: However, the intensity of the game does not seem to be as intense as the old man in the park. For robots, table tennis

The first mechanical claw! Yuanluobao appeared at the 2024 World Robot Conference and released the first chess robot that can enter the home The first mechanical claw! Yuanluobao appeared at the 2024 World Robot Conference and released the first chess robot that can enter the home Aug 21, 2024 pm 07:33 PM

On August 21, the 2024 World Robot Conference was grandly held in Beijing. SenseTime's home robot brand "Yuanluobot SenseRobot" has unveiled its entire family of products, and recently released the Yuanluobot AI chess-playing robot - Chess Professional Edition (hereinafter referred to as "Yuanluobot SenseRobot"), becoming the world's first A chess robot for the home. As the third chess-playing robot product of Yuanluobo, the new Guoxiang robot has undergone a large number of special technical upgrades and innovations in AI and engineering machinery. For the first time, it has realized the ability to pick up three-dimensional chess pieces through mechanical claws on a home robot, and perform human-machine Functions such as chess playing, everyone playing chess, notation review, etc.

Claude has become lazy too! Netizen: Learn to give yourself a holiday Claude has become lazy too! Netizen: Learn to give yourself a holiday Sep 02, 2024 pm 01:56 PM

The start of school is about to begin, and it’s not just the students who are about to start the new semester who should take care of themselves, but also the large AI models. Some time ago, Reddit was filled with netizens complaining that Claude was getting lazy. "Its level has dropped a lot, it often pauses, and even the output becomes very short. In the first week of release, it could translate a full 4-page document at once, but now it can't even output half a page!" https:// www.reddit.com/r/ClaudeAI/comments/1by8rw8/something_just_feels_wrong_with_claude_in_the/ in a post titled "Totally disappointed with Claude", full of

At the World Robot Conference, this domestic robot carrying 'the hope of future elderly care' was surrounded At the World Robot Conference, this domestic robot carrying 'the hope of future elderly care' was surrounded Aug 22, 2024 pm 10:35 PM

At the World Robot Conference being held in Beijing, the display of humanoid robots has become the absolute focus of the scene. At the Stardust Intelligent booth, the AI ​​robot assistant S1 performed three major performances of dulcimer, martial arts, and calligraphy in one exhibition area, capable of both literary and martial arts. , attracted a large number of professional audiences and media. The elegant playing on the elastic strings allows the S1 to demonstrate fine operation and absolute control with speed, strength and precision. CCTV News conducted a special report on the imitation learning and intelligent control behind "Calligraphy". Company founder Lai Jie explained that behind the silky movements, the hardware side pursues the best force control and the most human-like body indicators (speed, load) etc.), but on the AI ​​side, the real movement data of people is collected, allowing the robot to become stronger when it encounters a strong situation and learn to evolve quickly. And agile

ACL 2024 Awards Announced: One of the Best Papers on Oracle Deciphering by HuaTech, GloVe Time Test Award ACL 2024 Awards Announced: One of the Best Papers on Oracle Deciphering by HuaTech, GloVe Time Test Award Aug 15, 2024 pm 04:37 PM

At this ACL conference, contributors have gained a lot. The six-day ACL2024 is being held in Bangkok, Thailand. ACL is the top international conference in the field of computational linguistics and natural language processing. It is organized by the International Association for Computational Linguistics and is held annually. ACL has always ranked first in academic influence in the field of NLP, and it is also a CCF-A recommended conference. This year's ACL conference is the 62nd and has received more than 400 cutting-edge works in the field of NLP. Yesterday afternoon, the conference announced the best paper and other awards. This time, there are 7 Best Paper Awards (two unpublished), 1 Best Theme Paper Award, and 35 Outstanding Paper Awards. The conference also awarded 3 Resource Paper Awards (ResourceAward) and Social Impact Award (

Hongmeng Smart Travel S9 and full-scenario new product launch conference, a number of blockbuster new products were released together Hongmeng Smart Travel S9 and full-scenario new product launch conference, a number of blockbuster new products were released together Aug 08, 2024 am 07:02 AM

This afternoon, Hongmeng Zhixing officially welcomed new brands and new cars. On August 6, Huawei held the Hongmeng Smart Xingxing S9 and Huawei full-scenario new product launch conference, bringing the panoramic smart flagship sedan Xiangjie S9, the new M7Pro and Huawei novaFlip, MatePad Pro 12.2 inches, the new MatePad Air, Huawei Bisheng With many new all-scenario smart products including the laser printer X1 series, FreeBuds6i, WATCHFIT3 and smart screen S5Pro, from smart travel, smart office to smart wear, Huawei continues to build a full-scenario smart ecosystem to bring consumers a smart experience of the Internet of Everything. Hongmeng Zhixing: In-depth empowerment to promote the upgrading of the smart car industry Huawei joins hands with Chinese automotive industry partners to provide

Li Feifei's team proposed ReKep to give robots spatial intelligence and integrate GPT-4o Li Feifei's team proposed ReKep to give robots spatial intelligence and integrate GPT-4o Sep 03, 2024 pm 05:18 PM

Deep integration of vision and robot learning. When two robot hands work together smoothly to fold clothes, pour tea, and pack shoes, coupled with the 1X humanoid robot NEO that has been making headlines recently, you may have a feeling: we seem to be entering the age of robots. In fact, these silky movements are the product of advanced robotic technology + exquisite frame design + multi-modal large models. We know that useful robots often require complex and exquisite interactions with the environment, and the environment can be represented as constraints in the spatial and temporal domains. For example, if you want a robot to pour tea, the robot first needs to grasp the handle of the teapot and keep it upright without spilling the tea, then move it smoothly until the mouth of the pot is aligned with the mouth of the cup, and then tilt the teapot at a certain angle. . this

Distributed Artificial Intelligence Conference DAI 2024 Call for Papers: Agent Day, Richard Sutton, the father of reinforcement learning, will attend! Yan Shuicheng, Sergey Levine and DeepMind scientists will give keynote speeches Distributed Artificial Intelligence Conference DAI 2024 Call for Papers: Agent Day, Richard Sutton, the father of reinforcement learning, will attend! Yan Shuicheng, Sergey Levine and DeepMind scientists will give keynote speeches Aug 22, 2024 pm 08:02 PM

Conference Introduction With the rapid development of science and technology, artificial intelligence has become an important force in promoting social progress. In this era, we are fortunate to witness and participate in the innovation and application of Distributed Artificial Intelligence (DAI). Distributed artificial intelligence is an important branch of the field of artificial intelligence, which has attracted more and more attention in recent years. Agents based on large language models (LLM) have suddenly emerged. By combining the powerful language understanding and generation capabilities of large models, they have shown great potential in natural language interaction, knowledge reasoning, task planning, etc. AIAgent is taking over the big language model and has become a hot topic in the current AI circle. Au

See all articles