I’m working on a workflow for my object detection project (using YOLOv8 via Roboflow) where users upload photos through a mobile app for real-time inference. To maintain and potentially improve model performance over time, I’d like to incorporate a step where these user-uploaded images are securely stored and queued for manual annotation by my team, rather than relying solely on automated labeling or active learning, which could risk degrading the model if edge cases aren’t handled properly. My goal is to periodically retrain the model with this curated dataset.
Has anyone implemented a similar setup? I’m particularly interested in the optimal workflow configuration.
I’ve considered using the “Upload Batches” feature and the annotation queue, but I’d love insights on best practices or potential pitfalls—especially regarding data versioning and avoiding performance drift. Any advice on integrating this with Flutterflow (my app frontend) or suggestions for a robust pipeline would be greatly appreciated!
Active learning would be what I suggest here, but it seems like you’ve looked at it already. We do have active learning as a feature, but it’s also a general concept in computer vision of continually improving your model based on training with new images from the wild.
Within workflows, implementing active learning involves using the “Roboflow Dataset Upload” block. There, you can set logic (through “Continue If” or other blocks) and configure it. It will end up in your project’s Annotate board where you can review and correct (or add) predictions as needed.
Some config options that might be useful for you:
Predictions: lets you attach model predictions for you to correct and review later. Data
Percentage: lets you sample a random percentage of images.
Minutely/Hourly/Daily Usage Limit & Usage Quota Name: you can limit how many images get uploaded to your project on a timeframe basis.
The Quota Name let’s you add dynamic quotas so (ex: I want X images hourly for this camera, etc)
Labeling Batch Prefix & Frequency: By default all the images are uploaded to one batch on your Annotate page. You can customize it here and the Frequency option lets you create “buckets” of images on a daily/weekly/monthly basis
We’re always trying to improve our product so if there’s information about this that wasn’t clear in the app or if the feature doesn’t work for your use case, let us know!
Hi and thank you very much for the clear and quick response. One last question : In a basic action chain that includes: Inputs->Object Detection model-> Bounding box visualisation-> Label visualisation → Outputs, between which and which box is it more relevant to place the “Roboflow dataset upload” active learning feature?
Glad you found it helpful! Within that workflow, the Dataset Upload block can go anywhere after the Object Detection model (if you want predictions attached to the upload) or anywhere (if you don’t).
One thing to be mindful of and test thoroughly is that the two visualization blocks you use produce an output image with annotated bounding boxes. You should make sure you don’t accidentally upload visualization images.
With that in mind, you might want to place it right after the model block to be safe, but if you configure it correctly, it shouldn’t matter.
