Hello All,
I’m trying to create an object detection model that can detect a custom made robot from an aerial image. For this project, it is crucial to know the orientation of the vehicle, therefore the model will be based off of the YOLOv5-OBB repository.
To build the model, I’ve followed this guide from Roboflow.
I was able to get the model up and running, but wasn’t happy with my results. After adjusting parameters like batch size and epochs I got slightly better results, but the bounding box wasn’t as tight to the robot as I would like.
I started to look into why this may be the case and I noticed that the Roboflow annotations when exported aren’t as accurate when I drew them.
The technical term for the YOLOv5-OBB annotation format is called “DOTA”. DOTA works by storing pixel coordinates of the vertices of the bounding box in a text file starting from the top left corner heading clockwise.
I’ve created a small Python script that verifies the Roboflow annotations by drawing a bounding box over the object by using the pixel coordinates found in the exported DOTA annotation text file.
To use it, just change the PATH variable to the path of your dataset. It will then store the results in a folder called “Annotations Drawn” In the root of the PATH. Please note that this script will only work if your model has only 1 class or object its trying to detect.
import cv2 as cv
import os
PATH = r"C:\Users\XXX\Desktop\Test script\Exported Annotation"
os.chdir(PATH)
Ipath = PATH + r"\train\images"
Apath = PATH + r"\train\labelTxt"
Spath = PATH + r"\Annotations Drawn"
if os.path.exists(Spath):
print("Save Directory already exists")
else:
os.mkdir(Spath)
print("Save Directory Created")
IMG_Path = os.listdir(Ipath)
Annot_Path = os.listdir(Apath)
for img in IMG_Path:
img = img[:-4] #remove file extension
for annot in Annot_Path:
annot = annot[:-4] #remove extension again
if img == annot:
#reassign extensions
img = img + ".jpg"
annot = annot + ".txt"
os.chdir(Apath)
#S/O to Geeks4Geeks
with open(annot, 'r') as file:
for line in file:
x1, y1, x2, y2, x3, y3, x4, y4, obj, difficulty = line.split()
file.close
#this algorithm only works for single class annotations
x1 = int(float(x1))
y1 = int(float(y1))
x2 = int(float(x2))
y2 = int(float(y2))
x3 = int(float(x3))
y3 = int(float(y3))
x4 = int(float(x4))
y4 = int(float(y4))
os.chdir(Ipath)
print(img)
image = cv.imread(img)
# cv.imshow('Car', image)
# cv.waitKey(0)
cv.line(image, (x1, y1), (x2,y2), (0,0,255), thickness=3)
cv.line(image, (x2, y2), (x3,y3), (0,0,255), thickness=3)
cv.line(image, (x3, y3), (x4,y4), (0,0,255), thickness=3)
cv.line(image, (x4, y4), (x1,y1), (0,0,255), thickness=3)
os.chdir(Spath)
cv.imwrite(img, image)
break
print("Done")
Here’s an example of the problem:
Annotation done in Roboflow:
Annotation verified by script:
As you can see, the bounding box coordinates when outputted isn’t as tight as the annotation I drew in Roboflow.
Initially I thought this was because the DOTA format uses only 4 points and when I used the polygon tool to draw my annotation I used more than 4 points (in an effort to get the tightest bounding box possible).
Therefore I generated a new version where I only used 4 points from the polygon tool to represent my annotation like so:
After confirming my annotation with script above I received:
As you can see the exported annotation is still wildly skewed and inaccurate.
I’ve tested multiple permutations changing the augmentation steps and enabling auto rotate (which made the problem way worse as the annotation would be completely shifted like so: )
No matter what changes I made the annotations would still show skewed although resizing the image to a lower resolution reduced the skewing effect.
One thing I did notice is if the annotation is relatively axis aligned i.e. the bounding box is roughly 90 degrees, the exported annotation will remain tight.
Please do not confuse this with the rotation augmentation step, but rather if the object itself is rotated.
Notice in the above images, no augmentations or preprocessing was applied but because the object is rotated, the corresponding annotation is skewed.
When the robot is relatively axis aligned like so:
The annotations are as follows:
Roboflow:
Verified:
As you can see when the annotations are axis aligned the exported annotation remains accurate as well.
Would one of the devs be able to look into this and provide a fix possibly? Thank you!
I think one solution would be to allow users to draw a bounding box that is perfectly a rectangle and then rotate it. Then just export the vertices of that rectangle.
TLDR: When an object is rotated, the exported annotation is skewed and looks like a rhombus.
EDIT: The forum won’t let me attach multiple images so I’ve removed them.
