Label overlapping objects to keep them from being incorrectly classified?

I’m new to annotating images and have a question. I found (using a very small training set) that the generated model appeared to be classifying parts of my images (that were not labeled) as belonging to the most common labeled class in my images. These incorrectly classified “objects” also tend to be contained w/in the bounding box of the labeled areas (I took pains to excluded them whenever possible).

Here’s an example, the numbered circles are not labeled but the generated classifier is often classifying them as one of the labeled categories:

I was wondering if it might be helpful to add labels for all the numbered circles to help isolate them from the desired categores?

In my subsequent reading, it looks like labeling the numbered circles wouldn’t help this situation. By, not labeling them they are effectively negative examples.