One day after Pika launched a new audio and video synchronization feature, the rollover video is here

Pika's new sound effect function "Sound Effects" has both big surprises and small "scares".

The emergence of Sora made Vincent video models and applications popular. However, most of the videos generated by such models are silent. Therefore, people began to explore “dubbing” for AI-generated videos.

Representatives in this field include AI voice cloning startup ElevenLab, which previously generated dubbing for Sora’s demonstration video, and the sound effects and video images were almost incompatible.

Now, Pika, the video generation startup that once detonated the AI ​​circle, finally has a new move - the launch of Sound Effects, which can seamlessly add sound effects to videos generated by Pika .

Pika has two ways to generate sound effects: one is to give a prompt to describe the sound you want to hear; the other is to automatically generate sounds directly based on the video content .

As for how well the sound effects match the video content? Let’s take a first look.

However, Pika’s Sound Effects feature is only open for testing to Super Collaborator and Pro users. Ordinary users can only wait any longer.

Currently, Pika has officially released many demos.

Experience address: https://pika.art/

For example, the sound of sizzling barbecue can be seen vaguely The smoke that rises from time to time, the high-definition picture and the realistic sound effects make people feel crispy, salty, and spicy rushing into the tip of the tongue:

If you want to change your voice , no problem, click the Redo button below the video, and in the dialog box that appears, enter the audio description you want:

Little frog lying on the lotus The leaves are croaking, and if you listen carefully, you can also hear the chirping of cicadas in the distance (but the generated video looks very flawed, for example, the lotus leaves are red, and the frog has legs on its abdomen): Fireworks bloom in the sky, and the loud sound of firecrackers is emitted at the moment of blooming. The click is very accurate:

The above are just official examples. What is the effect of the generated audio? We We still have to see the real feelings of the first wave of users.

The first batch of trial experiences are released: there are surprises and disappointments

In Pika After the launch of the Sound Effects function, the first batch of trial users have already posted their AI videos with sound effects added.

For example, in the video of people in the rain generated by user @TomLikesRobots, he said that the sound effects seemed to match the video very well, adding a lot of atmosphere.

#For example, the sound effects generated by user @RetropunkAI are very interesting.

For example, user @AiFreak_tool added different sound effects to the generated video.

Of course, the sound effects generated by Sound Effects may not always fit the video theme and picture, such as the example below.

#                       Source: @TomLikesRobots

There is also user @DavidmComfort, who said that the generated sound effects are sometimes wrong, but after all, it is in the beta stage, so it is excusable.

Although there are occasional overturns, more people still expect Pika to open the API quickly.

The above is the detailed content of One day after Pika launched a new audio and video synchronization feature, the rollover video is here. For more information, please follow other related articles on the PHP Chinese website!