Custom Vision training and deployment options

Community Help
1

Hi,

I have project requirement to build object detection model using my organization’s dataset for manufacturing defects. I found Roboflow is suitable. I trained , deployed custom dataset and tried inference. I have following questions to proceed further:

  1. Our company image data is stored in S3 , how to train using S3?
  2. How to do MLOPS in Roboflow, for example automatic retraining of new data and deployment?
  3. I want to setup monitoring dashboard to see how many request hit the endpoint URL, model accuracy, data drift if any. How to setup model monitoring dashboard
  4. I want email trigger, if model performance slow or time out, how to setup email alert?

Please let me know.

Thanks,
Parthib.

2

Hey, Parthib!

Glad you’re finding good opportunity with the tooling. Some thoughts for your questions:

  1. You can upload and sync data from S3 a few ways, and the best way may depend on the volume of data you’re working with. You can use signed URLs to upload directly or download/upload. Here’s a few documented approaches: AWS S3 Bucket | Roboflow
  2. Yes. The best way to do this is triggering a train job to kickoff via API. For example, once a dataset has greater than X new images, you create a new trained model. A common way to do that is with the Python SDK: Train a Model | Roboflow
  3. Every trained model has model monitoring included (Model Monitoring | Roboflow). You can also pull model monitoring statistics via API to write them to your own dashboards: Inference Result Stats | Roboflow
  4. You can setup alerts based on model monitoring capabilities in (3). Typically, these alerts are if there’s a prediction that is of a specific confidence or drift in results.

Our team would be happy to show you how to setup many of these capabilities directly. Please feel free to write joseph.nelson [@] roboflow.com to get a guided walkthrough.