Does an instance segmentation model benefit from both bounding box and polygon annotation data?

Hello,
I am working on an instance segmentation project that I plan to train on Yolo8.
I have about 140 images that have been well annotated with polygons (a lot of work).
I have also found on the Roboflow Universe hundreds of annotated images of the same type of object but which have been annotated with bounding boxes.

Does it make sense clone those images and to train the model with both bounding boxes and polygon data? Or do I have to re-annotate the bounding box images with polygons if I want to use them? I am a beginner here, so please explain accordingly. Thanks for any reply.

Hi @dez

Using object detection bounding boxes for training an instance segmentation model will most likely not produce good results. Using polygon annotation data on an object detection model, on the other hand, would likely not hurt. (We are doing a blog post on this topic, so stay tuned)

You could consider changing to an object detection project instead if you don’t need polygon detection results.

If you do, you can use our tools like Smart Polygon to label polygon images faster

Thank you for the reply.

That makes sense. It is what I imagined, but I wasn’t positive since I am just learning.
I do need instance segmentation, so I have been doing the polygons.
As I am sure you know, it is a lot of work.
My images tend to have many items of interest on each frame (50-300). I have found a fast way to work is to create bounding boxes on everything, and then use the Roboflow SAM tool to convert them all to shapes, and then follow by cleaning them up.