Hi, I’m looking for tips for improving my model’s performance. I’m asking a lot of my model; to identify open blueberry flowers in a complex, 3-dimensional space. I’ve annotated roughly 280 images so far, but I have thousands. The more I annotate, the better my precision gets, but recall is lagging behind. In my most recent version, I got:
mAP@50 = 66.3%
Precision = 79.4%
Recall = 68.3%
F1 = 73.4%
Looking for general tips to improve my model. I want to label every flower possible to improve recall, but I also want to avoid blurry background flowers and flowers that have fallen to the ground. Tips on how to strike a balance?
Hey - great project you have going! Not being a blueberry flower expert myself, I have a few questions and possible suggestions.
Below is a screenshot of the middle of your image. I have put yellow boxes around items that to me also look like the same flowers you have inside the blue bounding boxes. (And there are still more I did not circle.) My worry is that if they confuse me, they may also be confusing the model.
Let me know if there is a reason those are not identified. But pending that answer, I have two guesses to offer up:
If those are flowers you just have not annotated, then by not doing so you are confusing the model. It does not know how to decide between those flowers and the ones you did circle. So in this case, I would go through all images and annotate those with a box as well and then train your model again.
However, if you do NOT consider those blueberry flowers, I’d first of all be curious what they are just to know for fun , but then I would say to go ahead and put a box around those and name them whatever they are. Then when you train the model, it will know that those specifically are NOT flowers and it will do better at finding what you want. It might just be confused right now.
Those objects that you’ve labeled are indeed blueberry flowers, but the goal of this project is to identify open blueberry flowers (i.e., accessible floral rewards for honey bees and other pollinators. A closed flower isn’t a good baseline to standardize pollinator floral visitation; that metric is usually expressed as #pollinators/#open flowers).
Do you recommend creating a second class which labels unopened flowers?
I see how confusing that can be, with an open flower and unopened flower looking so similar.
Makes sense and you are absolutely correct! Create that second class for unopened_flowers and I think you’ll start to see some improvements. Keep us posted on the progress out here and if you hit any more roadblocks!
, but then I would say to go ahead and put a box around those and name them whatever they are. Then when you train the model, it will know that those specifically are NOT flowers and it will do better at finding what you want. It might just be confused right now.