What are the advantages and disadvantages of manual annotation versus automatic annotation of objects in images, especially in the context of machine learning and computer vision?
Hi @tych_mbaz - good question.
First, there are a range of ways you can automatically annotate images:
- Roboflow’s open-source Autodistill repo uses heavy foundational models to create context-specific labels that can then train lighter models that are fit for deployment.
- Roboflow offers Model-Assisted Labeling, which uses a custom model (trained on Roboflow or outside the platform) to label new images.
If you are just getting started, it is always better to annotate the first few images by hand. This gives you a sense of your images and allows you to better formulate the right labeling taxonomy before you spend time automating it.
Once you are ready to scale, automated methods are often useful but have drawbacks. Autodistill is best for common objects that foundational models are familiar with, like “bottle”, “car”, or “person”. It won’t work well on very specific items. For example, if you were trying to label a specific kind of gear on an engine in a factory setting, Autodistill is likely not going to work for you.
I personally use Model-Assisted Labeling in all of my projects. It allows you to build a model iteratively (label 50 images, train a model, see how Label Assist performs, label 200 more images) while getting a great hands-on sense of how the model performs. If Label Assist works poorly on a specific class, I make sure to over-sample that class in my next batch of images. However, you require a human in the loop. Definitionally, the worse your model is, the worse this method will perform. Therefore there is a bit of tension: you need additional images the most when the model is weak and not nearly as many images when the model is strong!
For these reasons, many of Roboflow’s customers opt to outsource labeling. It’s quite cheap - the rate we currently offer to paying customers is $0.03 per bounding box label and $0.06 per polygon label.
Let me know if you have any questions on any of these options!