Describe your question/issue here! (delete this when you post)
Project Type: Auto Annotation Segmentation SAM3
Operating System & Browser: Windows / Chrome
**Project Universe Link or Workspace/Project ID: unavailable
I ran an auto annotation on a hand and a ball on 500 images to test SAM3. Now in the review section I have 500 images to review. Some of the segmentations that were made are slightly off and I want to edit the mask. Whenever I click on any part of the image to edit the mask with the Drag tool, it automatically switches over to the brush tool. How do I edit the auto detected masks before approving? I tried breaking the image into polygons so I could move the points, but still didn’t do anything. Any suggestions? I want to make edits to the ball and the hand classes. Thanks!**
For additional context, I also watched this youtube video and this guy appears to have no trouble just clicking on the image and changing it. Why do I get switched over to the brush when I click on the image? https://www.youtube.com/watch?v=SDV6Gz0suAk
Hey @SamEhrlich , the SAM3 stuff is so new I had to jump in myself and see what you were talking about. Someone from Roboflow might chime in with more info, but I did have the same experience - unable to do edits to the existing mask that SAM3 created. But I’ve got some thoughts for you.
SAM3 appears to be generating a pixel-level mask. It does seem that’s not editable after it’s generated. Although when I think about making that editable - oof! Going in to edit masks at the pixel level on hundreds of images!? You know what I mean? (Now, even if it’s not a bug, maybe it’s a feature request. But I personally would probably opt for the below to update masks like this that need a fix.)
In the meantime, the quickest workaround I found for items with a mask you want to edit is probably to replace the mask:
click on the target object and it will usually guess a really good starting mask
now you have two options
polygon - go with a more basic polygon mask. Click “Finish” if it’s close, THEN you immediately get the option to drag points around as needed. (NOTE - these polygons CAN be moved around by the (D)rag tool after they are created.)
pixels - you can go pixel level like the old mask (see below where you have to pick that option). It’s the same end result as before though - once created you can’t manually edit which pixels are included in the mask. However - after initially selecting the object, you can click outside the mask to add in missing pieces, or click in the mask to remove pieces that don’t actually belong to the object. And you can do this with some fairly good precision as needed. (This is also a quick way to fine-tune the polygon method offered in this window.)
Hopefully that at least helps you move ahead on your project. I did glance at the YouTube video - looks like they were working with bounding boxes and those do respond to the (D)rag tool.
One final note in case you weren’t aware - you can move detections to be “in front” or “behind” other detections. This layering helps when trying to select overlapping items, like you have. You just right click on a detection to move it (per the image below). You can see here there is an object on the keyboard so I moved that keyboard detection backward to let me click on the other detection more easily.
Thank you so much for replying! I am glad I’m not the only one with this issue. I figured these masks would be smart polygon type objects that can be moved around by different points.
I need to separate the ball and hand a little better to fine tune the SAM3 model. I will probably end up deleting bad masks and remaking them with the polygon tool myself. Do you think there will be a feature added to adjust the masks in the future? I am pretty new to Roboflow, so if there is a way to get in contact with anyone to request this type of feature it would be great!
I spent quite some time this weekend working with these tools, and I did find the move to front or back useful. I was trying the “split into polygon” as well but that did not work as the masks are on the pixel level for now.
@Automatez also if you have any socials I would love to have a contact to bounce ideas off of while I am still learning. If we can message through here, or if you have Twitter we can communicate there as well! I believe my twitter is in my bio/github maybe? Thanks!
In reality, SAM3 generates masks and polygons, unlike other instance-segmentation models that use polygons.
For editing these masks, we currently have the Brush Tool, which works like a paintbrush that lets you add or remove parts of the mask.
I believe it’s not very clear today that when you click on a mask, the Brush Tool is automatically activated, and you can start editing the mask directly by drawing on the image.
We have upcoming improvements planned for masks, and I’m happy to hear your feedback and suggestions!
Am I able to add and remove parts of the mask using the brush tool or only add? Also how accurate is the brush? I know I can change the size. An example that I am working through currently is the splits between fingers is masked as part of the hand, and I want to remove parts of the mask to better outline the hand.
Thanks for providing insight on the brush tool! I was wondering why it kept defaulting to the brush..
