I’m working on instance segmentation for a series of satellite images. I’m annotating geographic features that sometimes aren’t fully visible due to image capture limitations.
I’d like to train three classes:
- Object fully in frame
- Object present but partially out of frame
- Intentionally missing object (null case)
The first class is straightforward. For the second class, I’m wondering if there’s a way to annotate “open” where the polygon extends to the edge of the frame where the object is detected, but remains open to the edge of the image. This way, the training data will know that the object should be detected as the second class.
Lastly, do I simply upload the images without annotating the null case?
I may be going down a bad path, so if there’s a better way I’m open to suggestions.
Sup! There’s so many approaches to projects like these so there’s never a bad path - just a bunch of paths you have to try to find the best!
It would help the community help you if you could do two things - one is explain a bit why you want to distinguish “fully in frame” vs “partially out of frame”. Are you trying to find cases where you need to retake or get a better image of that region? Or are you just worried about missing those partials? And then adding a pic example of both situations might help too.
But I’ll throw some stuff out here. Not knowing your familiarity with object detection, I’ll first say that it is meant to find objects regardless of how much can be seen. So you are giving the example of like half a lake at the edge of a picture. But there’s also the scenario of half a lake blocked by a bunch of trees. The detector does not really care why the partial view happens so a good data set has both of those, as well as “complete” lakes, and it learns that those are all “lakes”. If you tried to train it on partial lakes at the edge of the image it could get messy if other things block a lake in the middle of the image, etc. So all that being said, here are my answers:
yes, easy
I would just annotate every lake as a “lake”. When they are at the edge, you just drag the box to the edge. When they are occluded in the middle, still put the box around the entire area you expect the lake is. With enough data it will learn to find all the lakes. Then you could do a SECOND model which is a classifier. You feed each lake detection (just the area in the bounding box that the model generates) into a classifier that is trained to detect “full lake” and “partial lake”. You should have more success when it’s able to focus like that, if indeed you really need to identify when you are getting a partial at the edge of an image.
Close! You definitely want to upload a bunch of images (maybe 10% or so of the total) that have no lake at all. Yes, there’s nothing to annotate BUT in Roboflow you actually need to mark them as “null image”. These nulls helps train the model even better than only images with lakes. (You can see the icon at the far bottom right of the image below, which doesn’t have a lake. )
The last thing I’ll say is that if I randomly guessed correctly that you’re looking for lakes you gotta give me a shoutout for that! Ha! Best of luck!
Sadly, your guess of lakes isn’t spot on, but a great example of what I might need. I want to know if the object (a fullLake is in view) and if not need to know if a partialLake is in view. Idea being that if a partialLake is found I can zoom out, far enough to get the full outline of the lake.
While I believe I could use confidence as a value to key off “might need to zoom out” I was hoping to classify partialLake so I had a simple bool:
if predictions.contains([partialLake]) {
zoomOut
captureAndReprocess
}
So I’m less worried about the case of something being “in front” of the lake, but rather that complete object (lake) not being completely in frame. In this example I provided about 50% of the image, but in practice it might be just about 100m within the image frame.
RE: No lake case, great! I did that. Thanks for the tip!
Dang! The lake guess would have been a nice win. I think adding the classification model step would get you your boolean. See this blog example where they already had zoomed in on juice boxes and they are now just classifying as “acceptable” vs a few defect states. You would just have the two classes though - “lake” and “partial_lake” - to train it on. How to Train and Deploy a Vision Transformer (ViT) Classification Model
)
Best of luck!
