Using SAM 2 for Video Segmentation via Inference Package

🤝 Community Help
1

I’m trying to figure out how to use SAM 2 to segment a video using an image mask through the inference.models.sam2 package. I know there’s a function called propagate_in_video available when using the SAM 2 model downloading directly from source, but I’m looking for a similar feature when running SAM 2 via the inference package. I’ve reviewed the documentation here: segment_anything2.py and here: Roboflow Inference SAM 2 guide, but I haven’t found any instructions on performing video segmentation. Any guidance would be appreciated.

2

Hi @Dikshant_Shah!
Unfortunately, at this time, you’re right that inference sdk doesn’t support video segmentation with SAM 2.

Apologies for the inconvenience!

3

No worries at all. But does Roboflow workflows support video input for SAM 2 blocks?

4

Hi @Dikshant_Shah!
Unfortunately not at this moment. I cannot provide a timeline, but this is on the team’s radar.

5

Hi @Dikshant_Shah ,

Please keep an eye on inference-exp, we are currently working to add SAM2 support with video into it. Bare in mind this work is in experimental stage, so anything may change.

Thanks!

Grzegorz

6

Got it thanks.