I am using RF-DETR for smoke detection. I seem to have some multiple predictions for smoke (usually TP bounding box preds that are quite close to one another that also match the GT) for some of the images in my test set. I thought that one of the main strengths of DETRs was that they don’t need NMS, but I still get a lot of multiple predictions, especially at lower confidence thresholds. I was wondering why this might be and how to address this. Especially in my evaluation of the model? Thank you so much for the help!
It might help if you can share some sample images. My instincts are telling me smoke is going to be a tough one. It’s so variable that finding an entire united area of smoke in a single bounding box could be tricky in general. But seeing some pictures with results might help others solution this for you!
Hi, thank you for this! For instance, here, the RF-DETR model makes 3 predictions for the same smoke object. This is at the confidence threshold that results in the best F1 (at lower conf thresholds, there are more multiple predictions). The initial evaluation logic used for the YOLO baseline counts the remaining boxes as FP once a successful match has been made (and multiple predictions weren’t an issue since YOLO applies NMS).
Yes, I agree, perhaps the struggle is the amorphous nature of smoke and drawing bounding boxes around it becomes tricky. I guess I can just apply NMS on my own to have a meaningful and fair comparison with the YOLO baseline. Thank you!
(For some images, it makes 100 predictions, when unfiltered with confidence thresholds, so I believe it must be about the challenging nature of smoke)
Hi @mila88!
Love your use case here! I completely agree, part of the struggle is indeed the amorphous nature of smoke and drawing consistent bounding boxes.
Additionally, to help improve your model, I suggest implementing this model in a workflow and using a dataset upload block. This way you can properly annotate the images like this to help your model to continue improving.
This topic was automatically closed 21 days after the last reply. New replies are no longer allowed.