The generative artificial intelligence (AI) market is a billion-dollar market expected to grow to $1.3 trillion over the next 10 years. This is why major tech companies have invested millions of dollars in developing gen AI tools, like image and video generators.
Meta recently launched own its AI video generator called Movie Gen, which can create realistic videos with sound. Movie Gen is a “cast of foundation models,” each specialized for various tasks like video creation, editing, and animation. These models work together to generate video from text prompts, enhance video quality, edit, add visual effects, or produce realistic animations.
Haiper AI Text to Video Generation: How to Use? Is it better than OpenAI Sora?
What is Meta Movie Gen?
Meta’s Movie Gen is a text-to-video AI tool that can create custom videos and sounds, edit existing videos, and transform an image into a unique video using simple text inputs. By combining image, video, and audio modalities, Movie Gen enables users to express themselves in new ways and bring their artistic visions to life.
This generative AI tool can create videos up to 16 seconds long with 16 frames per second. Meta claims that it outperforms similar tools including OpenAI’s Sora, Kling 1.5, Gen3 by Runway, and Luma AI’s Dream Machine.
Specifications
Here are the specifications, parameters, and benchmarks for Movie Gen:
- A 30B parameter model for text-to-video generation.
- Generates 1080p HD videos, with a duration of up to 16 seconds at 16 frames per second (FPS).
- Pre-trained on 100M videos and 1B images.
- Capable of video personalization and editing.
- 30B parameters for Movie Gen Video.
- Pre-trained using Flow Matching, leveraging a vast dataset of videos and images for video generation and a large audio dataset for sound generation.
- Outperforms prior state-of-the-art models like Runway Gen3 and OpenAI Sora on video quality.
Image Credit: Meta
For more information on the model’s training, specification, and benchmarks, you can read Meta’s research paper here.
What Is OpenAI Sora? How To Use It?
Meta Movie Gen: Key Features
Here are the key features of Meta’s Movie Gen:
Realistic Video Generation
Meta Movie Gen is a 30B parameter transformer model that generates high-quality videos from text prompts. It can create 16-second clips at 16 frames per second. It excels at understanding object motion, subject-object interactions, camera movement, and creating realistic animations.
Personalized Videos
Movie Gen can also create personalized videos. By combining a person’s image with a text prompt, the model generates videos that include the reference individual while adding visual details. It ensures accurate human identity preservation and realistic motion in its personalized videos.
Precise Video Editing
Meta’s text-to-video generator also supports precise video editing, using both video input and text prompts. It can perform localized edits such as adding, removing, or replacing elements, and also make broader changes like modifying backgrounds or styles.
Audio Generation
One of the most distinguishing features of Movie Gen is that it creates video with sound. Movie Gen produces high-quality audio synced to video, including ambient sounds, effects, and music, based on optional text prompts. It can generate up to 45 seconds of audio and, with an extension technique, handle videos of any length.
Meta Launches Movie Gen AI: Create Realistic Videos & Sound, Competing with Top AI Platforms
When will Meta Movie Gen be Available?
Movie Gen is not available for public use. The research team at Meta has announced that videos from the paper, benchmarks, and other materials are available for viewing on their research pages. However, there are no direct plans to release this advanced Gen AI model for public use.
The Bottom Line
Meta is working closely with filmmakers and content creators to receive feedback and improve their AI models. The focus is on enhancing the quality and capabilities of Movie Gen before potentially making it available to the public in the future.
What is Flux.1 Model? Know All About New AI Image Generator Tool
