Any way to stop the training in Roboflow dashboard? My training seems to be stalled at 300 epochs and nothing has happened for some time…
Model Type:
Roboflow 3.0 Instance Segmentation (Fast)
Checkpoint:
COCOn-seg
1 Like
I’ve done some digging into your question as I find it particularly interesting. I do believe there should be a way to stop live training from the dashboard, however from what I have found, direct training from the Roboflow Dashboard does not offer the ability to do so.
Since you say that this has been going on for some time, we can assume it is not just a half or momentary pause or lag, the training process has genuinely halted.
I tried to run a training myself and see if I could stop it, however I could not find a way to do so. I took your question and put it through the Lenny Roboflow generative AI built on Roboflow data and documentation and to summarize what I got back, there could be a number of issues causing this to occur however the next steps for you are quite simple. Given my understanding Roboflow does not support stopping the training directly from the dashboard, we can assume the next step would be to contact Roboflow support and create a ticket for your specific issue so that the developers can resolve this and create documentation for future potential cases.
Here is the exact response from the AI given your issue:
***Based on the information provided in the extracts, there doesn’t seem to be a direct way to manually stop a training process from the Roboflow dashboard once it has started.
If your training seems to be stalled, it might be due to factors like the size of your dataset, the complexity of the model, or network issues.
If the issue persists, I would recommend reaching out to Roboflow’s support for further assistance. They might be able to provide more specific guidance or investigate if there’s an issue with the platform.
For more details on how to train your model in Roboflow, you can refer to the Roboflow documentation.***
My suggested solution: Since the training has a checkpoint we should terminate the current training, review the annotations, labeling, or any potential areas for bugs, and reinitiate the training from the checkpoint.
Following this documentation you should be able to start the training again from where you left off:
Hope this helps and good luck!
@Hart_Woolery When you say “stalled,” did the training complete?
Note that hosted training will do as many epochs until the model loss does not continue to improve. On the last epoch of training, the model is compiling final statistics like performance on the validation and testing set. This may give the appearance of it being stalled (and is good UX feedback) even though the model is continuing its work.
@Hart_Woolery can you add roboflow support to the project and link me the project?
I’ve granted Roboflow Support access to workspace
@Hart_Woolery thanks, can you link the project to me?
@Hart_Woolery it’s fixed now
2 Likes