We have a trained instance segmentation that we’re using via hosted inference, using workflows. In workflows, you can set IoU, but it’s not clear how it works. The documentation points to this: How to Code Non-Maximum Suppression (NMS) in Plain NumPy seems to refer to object detection specifically.
However I’m seeing a few cases where my model is missing a really obvious roof face, while the adjacent roof face is well detected. I wonder if the IoU setting uses bounding boxes, or the pixel masks themselves.
Especially when the roof faces are diagonal on the image, the bounding boxes of the detection may overlap quite a lot even when the detections themselves do not overlap at all, and this setting may filter out too many real, valid detections!
I have experimented with setting the IoU value to be high (e.g. 0.95) but then we do get quite a few more genuinely overlapping detections. I’m hoping the model can be smart enough to actually compare the masks rather than the bounding boxes.
I understand that doesn’t solve your problem, but hopefully confirming at least is somewhat helpful?
We don’t have this on the roadmap right now at roboflow, but I’ll note it as a feature request in the backlog. Since inference is open source, we’d be happy to have you make a PR if it’s something you want to implement yourself.
Hi Thomas, thanks for the response, definitely helpful. I guess we can set the value to 0.99 (1 seems to error for some reason) and then do the filtering on our end afterwards. We’ll take a look at doing a PR too.