I want to train a SINGLE (latency/speed purposes) yoloV8 model to detect clash royal troops. I will then take the centre position of the bounding box and troop type to feed into a decision making model which will learn to play the game.
To simplify my question, the model looks for 2 classes (red musketeer and blue musketeer).
The two classes look identical apart. The only way to tell them apart is by a tiny red/blue feather in their hat or the red/blue troop Lv icon above their heads.
Neither option is reliable as they might get blocked by other troops (the red icon also sometimes doesn’t show up and it can get confused with other troops).
I don’t want to create a box around the troop + Lv icon because there will be an inconsistency with the position of the troop (hurts decision model as lv icon doesn’t always appear).
If I have, say, 20% of training data with lv icon will it:
draw a tight crop around the troop.
be able to learn that red icon = red troop as a backup when other features are blocked?
So I end up with the best of both worlds. Thanks a lot for any help!
(the like tiny blue triangle below indicated blue musketeer)
That sounds like a fun project! So many ways to handle all this stuff but here are my thoughts.
I would include in your training data troopers annotated with and without the Lv icon. Keep the bounding boxes tight around both. That way it learns both scenarios can represent a given trooper. It won’t consider pixels outside your annotations as that would lead to endless possibilities.
If it struggles to see both as the same class detection, just create 2 classes - red_withLv and red_noLv - and then rename both classes to “red_trooper" before taking actions.
I also wonder if adding a tracking algorithm (like ByteTrack) would help with those times the color disappears. If I remember right, there’s usually pretty good visibility right at troop deployment so if you start tracking it might minimize the occlusion problem.
And then this one is a long shot, but in the game do you have to select which fighters to deploy on the screen before you deploy them? If so, and that selection process makes it clear who you picked, you could add a class to detect “selected_red" and use logic to assign that selection to the next new troops deployed. (Though I understand this wouldn’t work for the opponent’s deployment.)
I’m sure others out here will have some great advice as well. Have fun and good luck!
