Custom Vision training and deployment options

Hi,

I have project requirement to build object detection model using my organization’s dataset for manufacturing defects. I found Roboflow is suitable. I trained , deployed custom dataset and tried inference. I have following questions to proceed further:

  1. Our company image data is stored in S3 , how to train using S3?
  2. How to do MLOPS in Roboflow, for example automatic retraining of new data and deployment?
  3. I want to setup monitoring dashboard to see how many request hit the endpoint URL, model accuracy, data drift if any. How to setup model monitoring dashboard
  4. I want email trigger, if model performance slow or time out, how to setup email alert?

Please let me know.

Thanks,
Parthib.

Hey, Parthib!

Glad you’re finding good opportunity with the tooling. Some thoughts for your questions:

  1. You can upload and sync data from S3 a few ways, and the best way may depend on the volume of data you’re working with. You can use signed URLs to upload directly or download/upload. Here’s a few documented approaches: AWS S3 Bucket | Roboflow
  2. Yes. The best way to do this is triggering a train job to kickoff via API. For example, once a dataset has greater than X new images, you create a new trained model. A common way to do that is with the Python SDK: Train a Model | Roboflow
  3. Every trained model has model monitoring included (Model Monitoring | Roboflow). You can also pull model monitoring statistics via API to write them to your own dashboards: Inference Result Stats | Roboflow
  4. You can setup alerts based on model monitoring capabilities in (3). Typically, these alerts are if there’s a prediction that is of a specific confidence or drift in results.

Our team would be happy to show you how to setup many of these capabilities directly. Please feel free to write joseph.nelson [@] roboflow.com to get a guided walkthrough.