Segmentation is determining which pixels in an image correspond to an item. It is useful for activities like photo editing and scientific imaging analysis. New AI-enabled picture editing features in Meta’s apps, such as Backdrop and Cutouts on Instagram, were inspired by Meta’s initial Segment Anything Model, which was launched last year.
Diverse applications in science, health, and many other fields have also been sparked by SAM. For instance, SAM has been applied in the medical industry to help diagnose skin cancer and in marine research to segment sonar images and study coral reefs. It has also been utilized in satellite imagery analysis for disaster relief.
Also Read: What is Open Source AI? How is it Good for the World, Developers and Meta?
These features are expanded to include video with Meta’s new Segment Anything Model 2 (SAM 2). Any object in an image or video can be segmented by SAM 2, and it can follow an object in real time across all of the frames in the movie.
Since segmenting video is far more difficult than segmenting photos, existing models have not been able to do this. Videos allow objects to appear and disappear quickly, as well as to be hidden by other objects or scenes. Many of these issues were resolved when Meta constructed SAM 2.
Meta thinks their research can lead to new opportunities, including simpler video generation and editing, as well as the creation of new mixed reality experiences.
Also Read: Meta unveils its largest ‘open’ AI Model to date
Additionally, SAM 2 could be used to monitor an object of interest in a video, facilitating quicker annotation of visual data for computer vision system training, such as that which is employed in autonomous cars.
Additionally, it might make it possible to choose and interact with items in novel ways during live videos or in real-time.
In keeping with Meta’s open scientific philosophy, they are making their SAM 2 research available so others can investigate additional features and applications. It would be interesting to see what applications of this research the AI community makes.
Also Read: How to use Imagine Me? Meta AI Tool for Selfie (Text to Image) Generation
