Long time to generate new dataset version

Community Help
1
  • Project Type: object detection
  • Operating System & Browser: windows, chrome
  • Project Universe Link or Workspace/Project ID: Logo detection

I’m trying to generate a new dataset version based on the images I already labeled, but it’s taking to much time.

  • Does removing old versions speed up the process ?

Thanks.

2

I’m sorry, but your question seems incomplete. You’ve mentioned “Project Type: object detection” and “Operating System” but haven’t provided a specific question or context. Are you asking how to create an object detection project on Roboflow, or are you asking about the operating system requirements for using Roboflow? Could you please provide more details so I can give you a more accurate answer?

Hi! I’m Roboflow’s AI powered bot. This is an automated response powered using AI with knowledge from our docs, blog, knowledge base, and previous forum responses. If you find this to be a helpful answer, please mark it as the solution using the checkbox icon below. If this doesn’t solve your issue, please follow up with what you’ve tried/why, and the community will continue to chime in as usual.

3

@Lenny
How can I speed up the time to generate a new dataset version?

4

Removing datasets from the model training phase will take less time as there will be less epochs, as it decreases the size for the model, this would also result in faster responses from the model. However the less data it has the less accurate it may be depending on the quality of the data.

In summary, yes it would speed up the training process, it is also worth noting by removing training data you may forego accuracy of the newly trained model that the removed data may have provided. :+1:

There are many factors that go into the speed at which a model trains but here are the important ones below!
  1. Model Size: Depending on your custom dataset, the Fast Roboflow Train model should train roughly twice as fast as the Accurate Roboflow Train model (source).
  2. Dataset Size and Image Size: The larger the dataset, and the larger the images in your dataset, the longer it will take for your model to train (source).
  3. Training from a Checkpoint: Training from a checkpoint, a method known as Transfer Learning, can help to reduce training time. This method initializes your model training from a previously trained model, which can provide improved training scores. On the other hand, training from scratch, which does not employ Transfer Learning, will initialize your model training with randomized initial values for the model weights, which can take longer (source).
5

Thank you @Alejandro_Bermudez for the complete answer, but I was asking about roboflow time to generate a new dataset versin instead of the training part.
Once I have data annotated I have to generate a new dataset version. But since I have many versions the process of creating a new dataset was taking more time than usual.
So my question was, is the time of generating a new dataset version depends on the amount of versions I already have in th current project ? If yes, removing old versions will speed up the process of generating a new version in roboflow.
I tested removing old versions, after backuping them, and it actually speeded it up :smile:

1 Like