Explorer

Mark Zuckerberg Showcases Meta's Latest Video AI Tool, Segment Anything Model 2: Here's How It Works

According to Meta, SAM 2 AI model's capabilities extend beyond images to include real-time object tracking across video frames

Meta has introduced an upgraded video-editing-friendly artificial intelligence (AI) model, Segment Anything Model 2 (SAM 2), designed to handle intricate computer vision tasks. This announcement, made on Monday, marks a significant enhancement over the previous version launched last year, which was utilised in Instagram's Backdrop and Cutouts features. Meta boss Mark Zuckerberg took to Instagram to show how the new feature works in a fun video.

Here's what Zuckerberg posted: 

 
 
 
 
 
View this post on Instagram
 
 
 
 
 
 
 
 
 
 
 

A post shared by Mark Zuckerberg (@zuck)

ALSO READ: Meta AI Studio Will Help Creators Bring To Life Digital Companions To Drive Engagement. Indian Users Won't Be Able To Access It Yet

What Can SAM 2 Do?

SAM 2 boasts advanced capabilities, including segment identification and tracking within videos. According to Meta, the model's capabilities extend beyond images to include real-time object tracking across video frames, even in challenging scenarios where objects move rapidly, change appearance, or are obscured.

Mark Zuckerberg Showcases Meta's Latest Video AI Tool, Segment Anything Model 2: Here's How It Works

The new model builds on a transformer architecture and includes a streaming memory component, allowing it to process video segments efficiently. Meta emphasised that SAM 2 was trained on its extensive video segmentation dataset, known as the SA-V dataset, ensuring robust performance in various applications.

Meta highlighted that the initial version of SAM had diverse applications, such as aiding marine scientists in segmenting sonar images to analyze coral reefs, supporting disaster relief through satellite imagery analysis, and assisting in medical fields by segmenting cellular images for skin cancer detection.

As an open-source model, SAM 2's code and weights are accessible on Meta's GitHub page, licensed under Apache 2.0. This license permits research, academic, and non-commercial use, inviting widespread testing and innovation.

GIF of foot dribbling soccer ball, with SAM 2 highlighting and following the foot and ball as they move

Meta envisions SAM 2 facilitating video editing, AI-driven video generation, and enhancing mixed-reality experiences. Additionally, its object-tracking feature is expected to streamline the annotation of visual data, thereby improving the training of other computer vision systems.

GIF of man playing ping pong, with SAM 2 highlighting his head and following it as he moves

Overall, the introduction of SAM 2 underscores Meta's commitment to advancing AI technology and making sophisticated tools available for broader use within the tech and scientific communities.

View More
Advertisement
Advertisement
25°C
New Delhi
Rain: 100mm
Humidity: 97%
Wind: WNW 47km/h
See Today's Weather
powered by
Accu Weather
Advertisement

Top Headlines

JPC Constituted On 'One Nation One Election' Bill, Priyanka Gandhi, Anurag Thakur Among Members
JPC Constituted On 'One Nation One Election' Bill, Priyanka Gandhi, Anurag Thakur Among Members
Moment When Speed Boat Rammed Into 'Neelkamal' Ferry, Which Capsized Killing 13 In Mumbai — On Cam
Moment When Speed Boat Rammed Into 'Neelkamal' Ferry, Which Capsized Killing 13 In Mumbai — On Cam
'PM Modi Should Sack Amit Shah By Midnight If...': Congress' Big Attack On Centre Over Ambedkar Issue
'PM Modi Should Sack Amit Shah By Midnight If...': Congress' Big Attack On Centre Over Ambedkar Issue
Madhya Pradesh Govt Spent Rs 9.25 Lakh Per Day On Air Travel, Paid Rs 32.85 Crore For 666 Trips: Report
Madhya Pradesh Govt Spent Rs 9.25 Lakh Per Day On Air Travel, Paid Rs 32.85 Crore For 666 Trips: Report
Advertisement
ABP Premium

Videos

Priyanka Gandhi's bag creates a stir in politics, listen to what the Congress MP says in answer to the questions | ABP NewsParliament Session: Mallikarjun Kharge's allegation, 'Amit Shah insulted Baba Saheb Ambedkar' | ABP newsParliament Session: What did Shah say in Parliament after which Congress accused him of insulting AmbedkarParliament Session: 'Let me tell the 54-year-old youth who wants to change the Constitution...' Amit Shah

Photo Gallery

Embed widget