Meta shows AI model that can generate video with sound

Published
0

NEW YORK: Facebook owner Meta announced on Friday it had built a new AI model called Movie Gen that can create realistic-seeming video and audio clips in response to user prompts, claiming it can rival tools from leading media generation startups like OpenAI and ElevenLabs.

Samples of Movie Gen’s creations provided by Meta showed videos of animals swimming and surfing, as well as videos using people’s real photos to depict them performing actions like painting on a canvas. Movie Gen also can generate background music and sound effects synced to the content of the videos, Meta said in a blog post, and use the tool to edit existing videos.

In one such video, Meta had the tool insert pom-poms into the hands of a man running by himself in the desert, while in another it changed a parking lot where a man was skateboarding from dry ground into one covered by a splashing puddle.

Videos created by Movie Gen can be up to 16 seconds long, while audio can be up to 45 seconds long, Meta said. It shared data showing blind tests indicating that the model performs favourably compared with offerings from startups including Runway, OpenAI, ElevenLabs and Kling.

The announcement comes as Hollywood has been wrestling with how to harness generative AI video technology this year, after Microsoft-backed OpenAI in February first showed off how its product Sora could create feature film-like videos in response to text prompts.

Technologists in the entertainment industry are eager to use such tools to enhance and expedite filmmaking, while others worry about embracing systems that appear to have been trained on copyright works without permission.

Lawmakers also have highlighted concerns about how AI-generated fakes, or deepfakes, are being used in elections around the world, including in the US, Pakistan, India and Indonesia. Meta spokespeople said the company was unlikely to release Movie Gen for open use by developers, as it has with its Llama series of large-language models, saying it considers the risks individually for each model. They declined to comment on Meta’s assessment for Movie Gen specifically.

Instead, they said, Meta was working directly with the entertainment community and other content creators on uses of Movie Gen and would incorporate it into Meta’s own products sometime next year.

Published in Dawn, October 5th, 2024

Opinion

Editorial

LG malaise
16 Nov, 2026

LG malaise

THE repeated delays in local government elections in Islamabad expose the reluctance of the government to strengthen...
Smog diplomacy
Updated 16 Nov, 2026

Smog diplomacy

Polluted air crosses the border regardless of the state of bilateral relations.
Wild trophies
16 Nov, 2026

Wild trophies

THE elite fixation on housing wild beasts as status symbols is an invitation to tragedy. A farmhouse owner and a...
Muddled hierarchy
Updated 15 Nov, 2026

Muddled hierarchy

The jurisdiction of the two courts and a mechanism for settling disputes between them should be spelt out in law, not left to judges to fight over.
Growing crisis
Updated 15 Nov, 2026

Growing crisis

Calling population growth a national emergency is easy. Ensuring that the response reaches the people who need it is the real challenge.
Organised rackets
15 Nov, 2026

Organised rackets

ON the International Day against Transnational Organised Crime, observed today, Pakistan finds itself in a tight...