Meta Launches Movie Gen AI: Create Realistic Videos & Sound, Competing with Top AI Platforms

Facebook’s parent company Meta said it has developed a new artificial intelligence model, Movie Gen, that it claims can compete with tools from top media creation businesses like OpenAI and ElevenLabs. Movie Gen can produce realistic-looking video and audio segments responding to user suggestions.

Movies about animals swimming and surfing were among the examples of Movie Gen’s works that Meta supplied. Other movies featured humans executing real-life activities, such as painting on a canvas, and utilizing images of themselves.

In addition, according to a blog post by Meta, Movie Gen may be used to modify pre-existing videos and produce sound effects and background music that are synchronized with the video material.

In a couple of these videos, Meta used the tool to alter a parking lot where a man was skateboarding from dry ground to one with a splattering puddle by inserting pom-poms into the hands of the man running alone in the desert.

Also Read: Meta Quest 3S vs Quest 3: Check Specifications, Price, and New Features

According to Meta, Movie Gen videos have a maximum length of 16 seconds, while audio files might have a maximum length of 45 seconds. It released statistics from blind testing that demonstrate the model outperforms products from businesses including Runway, OpenAI, ElevenLabs, and Kling.

The news coincides with Hollywood’s struggles this year to use generative AI video technology, following Microsoft-backed OpenAI’s demonstration in February of its product Sora’s ability to produce feature-length videos in response to text commands.

While some entertainment industry technologists are excited to employ these technologies to improve and speed up filming, others are concerned about adopting algorithms that seem to have been trained on copyright works without authorization.

Concerns regarding the use of AI-generated fakes, or “deepfakes,” in elections around the globe, including those in the United States, Pakistan, India, and Indonesia, have also been brought up by lawmakers.

Also Read: Meta AI Products Update 2024: Check New Features and How to Use them?

As it has done with its Llama series of large-language models, Meta representatives stated that the company is unlikely to make Movie Gen available for open use by developers, stating that each model’s hazards are evaluated separately. They declined to respond to specific questions on Meta’s evaluation of Movie Gen.

Rather, they said that Meta was collaborating closely with the entertainment industry and other content producers on the application of Movie Gen and will integrate it into its products at some point in the upcoming year.

As stated in the tool’s blog post and research paper, Movie Gen was constructed by the company using a combination of licensed and publicly available datasets.

This year, OpenAI has been meeting with representatives and executives from the film industry to explore potential collaborations, including Sora; however, no agreements have reportedly been reached as of yet. Actress Scarlett Johansson’s accusation in May that the ChatGPT creator was using her voice for its chatbot without authorization heightened concerns about the company’s strategy.

Also Read: Meta’s Orion AR Glasses Debut: Advanced 70-Degree Field of View & AI Assistance

The studio that brought you “The Hunger Games” and “Twilight,” Lions Gate Entertainment, said in September that it would be providing AI startup Runway with access to its collection of movies and TV shows so that the latter could train an AI model. The model can be used by the studio and its filmmakers to enhance their work in exchange, it stated.