Going backwards and forwards removes old annotations?

When I use assisted labeling and click back through a set of images to annotate it seems to remove all the labels I previously made and redoes them. Is this what is happening or am I missing something?

It definitely does this. IMHO assisted labeling should never remove existing annotations.