Built my first model : There was an error running your Automatic Labeling test. Please try again

[silkyclouds]

Hi everyone,

I’m new to roboflow and trying to create a model to detect several type of objects using cctv sources.

I uploaded a set of approx. 9800 images and let the auto label feature operate its magic.

I then started to annotate the images and approved 378 out of these 9800 images.

Once done, I created a “new version” of my model and let it run :

I then moved my annotated images back to the unassigned section, and tried to start an auto label task using my improved model :

but I’m getting this error as soon I press the generate test results button :

" Error Generating Test Results

There was an error running your Automatic Labeling test. Please try again."

Am I not supposed to specifically use this process to re-annotate all the remaining images with the improved model, then approve the annotations again to improve it, re-train it etc. ?

Any help welcome !

Thanks!

[silkyclouds]

Seriously, no one ?

[Hunter-Lanier]

Hey! Super Cool Project! You are using the auto labeling exactly the way its supposed to be used, good job. I hate that things aren’t working for you. Lets see if we can get it sorted out.

Ive got a couple questions:

1. How did you train the model? was it trained locally and uploaded to roboflow? or was it trained with training credits?

the reason I ask is because there might be an issue with the model format transferring over into the auto label feature. Its still in beta so I could see that being an issue

2. Has it worked since this post? I know i’ve run into issues like this before and they just worked after some time. It could be an internal issue at the time, but isn’t present now. I just tested out the auto label feature on my dataset and it seems to be working correctly, so hopefully yours is too now.

like I said, auto labeling is in beta so its expected to have issues here and there, but I hope that we get it figured out. Your project looks awesome, and auto labeling will be a huge help.

[Jacob_Witt]

Hey @silkyclouds - if you’re still seeing errors, do you mind posting a screenshot of what you’re seeing with the dev console open (cmd+option+i on a mac), make sure to click to Console

[silkyclouds]

gents, thanks for your kind answers.

Following your answer, I tried it one more time and it now works.

It was failing right after I selected my trained model and clicked on the test button. with the previously providen error message.

But now, I’m requested to buy credits. This being a purely non-profit training, for studies purposes, I can’t pay for credit. unfortunately.

So I guess that I’m already done using roboflow to improve my model.

[Jacob_Witt]

Hi @silkyclouds - we’re running GPUs to provide this service and cannot offer unlimited use of Auto Label for free.

Students and researchers working on non-profit use-cases can apply for more usage at Contribute to the Computer Vision Community - I recommend you give it a look.

[silkyclouds]

Perfectly understandable.

I am working on a code available to anyone for free on github (noisebuster) and the idea is to try to add vehicle detection to it.

See my github profile.

I can read the following:

This offer is only available if your work is non-commercial and related to research, academia, or open source projects.

I have no .edu email address though.

[Jacob_Witt]

Your best bet is self-hosting Autodistill or your Roboflow model via our inference package.

[silkyclouds]

I guess I’ll have to use this on premise indeed. I don’t believe anyone that use your tool in a non-business environment can afford to spend 249$ monthly to train a model.

May I ask you if any of the suggested options above provides auto labeling assuming all processing will happen on my own GPU (I have a 3090 that should fit I guess?)

thanks again !

[Jacob_Witt]

Both options I shared can run on your own GPU

[silkyclouds]

Yes I am digging the docs now, seems quite easy to run on premise indeed. Thanks for letting me know about that ! (and too bad you guys don’t have cheaper subs for “private tinkerers”, as your webUI is great! )

[silkyclouds]

Hi @Jacob_Witt ,

I spent a few days checking the annotations, and ran a few trainings.

I’m here now (on roboflow) :

I did remove the “person” class as I am not in need of this type of object for my project, so I guess the next training will have higher values.

As I cannot generate the .pt model to use locally, I downloaded the dataset, and had to train it again using ultralytics tool.

Thing is, once I’m using the generated .pt file in my real-time analysis CCTV script, the results are terrible. cars are taken for trucks, I can see a LOT of vehicles like quite visible cars passing by without being detected…

I guess I’m doing something wrong here. what should I do to take advantage of the current map of my roboflow trained model, and apply it to my real-life cctv images ?

[silkyclouds]

I tried to use the custom model training snippet, but it wont accept my (verified and it is the right one) API key :

RuntimeError: {
    "error": {
        "message": "This API key does not exist (or has been revoked).",
        "status": 401,
        "type": "OAuthException",
        "hint": "You may retrieve your API key via the Roboflow Dashboard. Go to Account > Roboflow Keys to retrieve yours.",

Can you eventually help ?

[njmarraccini]

hey happy to help, going to send you an email!

[silkyclouds]

Thank you for the kind answer and interest @njmarraccini! As @Jacob_Witt told me, there is no option to use roboflow without spending 250$ monthly to train my model.

As I am not a business, but an individual, this is simply a the budget I want allow to this project.

So, I downloaded my dataset and I am now using Ultralytics to re-train my model on one of my home servers with a dedicated 3090, in YOLO11 format, with no issues.

Thanks again for reaching out !